The cynical amongst us would say that the big 5 storage vendors [...] ]]> I recently wrote an article for TechTarget that looked at the implementations of data de-duplication in primary storage arrays.  One of the things that stood out for me was the lack of de-duplication support in traditional (and some might say legacy) storage arrays.

The cynical amongst us would say that the big 5 storage vendors have no vested interest in introducing a technology such as de-duplication.  If dedupe rates can reach those of secondary storage products (i.e. 90% or more) then storage vendors are going to be selling way less storage than they do today – not the most desirable scenario.  However I wonder if technology plays more of a role here than pure financial considerations.

The Netapp Effect

Netapp introduced A-SIS (advanced single instance storage) into their filer product range in 2007.  Although continually berated by other vendors as performance afflicted, A-SIS does work and does produce savings.  Again, the cynical may say that Netapp needs to have some space saving technology in place, bearing in mind the inefficiency of WAFL, however it was still a bold move by the company and five years on, none of the other top 5 have followed their lead (EMC have talked the talk but failed to deliver as yet).

Perhaps through serendipity, Netapp have implemented an architecture that works well with de-duplication.  The 4KB block structure, write-new style of WAFL makes technologies such as thin provisioning and de-duplication relatively easy to implement (although it also causes headaches in delivering other functionality such as decent tiering).

On the other hand, other array architectures would find it very difficult to implement de-duplication.  EMC VMAX, and Hitachi VSP (aka HP XP24000) still retain their legacy LUN structure, onto which they layer wide striping pools and thin provisioning.  The block size in these architectures will be a limiting factor.

Design in Mind

That brings us to the SSD-based array vendors.  These companies have a vested interest in implementing de-duplication as it is one of the features they need to help make the TCO for all SSD arrays to work.  Out of necessity dedupe is a required feature, forcing it to be part of the array design.

Solid state is also a perfect technology for deduplicated storage.  Whether using inline or post-processing, de-duplication causes subsequent read requests to be more random in nature as the pattern of deduplicated data is unpredicable.  With fixed latency, SSDs are great at delivering this type of read request that may be tricker for other array types.

Mainstream or Not?

Will de-duplication become a standard mainstream feature?  Probably not in current array platforms but definitely for the new ones where legacy history isn’t an issue.  There will come a time when those legacy platforms should be put out to pasture and by then de-duplication will be a standard feature.  When that will happen will have to be the subject of another post.

Related Articles You May Be Interested In

• Solid State Storage Symposium 25th April 2012
• Lefthand SSD Array – Podcast
• EMC Releases All Flash VNX
• Solid State Arrays: SolidFire
• Solid State Arrays: Pure Storage Inc
• Who Will Be The First Solid State Array Vendor To Be Acquired?

I’ll be there both days and I’m pretty busy with vendor briefings, but if you are going, drop me [...] ]]> This week I’ll be attending IP Expo, which has probably become the UK’s biggest IT event.  You can find more details out at the IP Expo Website, but if you’re quick, entry is free.

I’ll be there both days and I’m pretty busy with vendor briefings, but if you are going, drop me a note and we can meet up over a coffee (or beer in the evening).  Remember, Steve Wozniak is doing the keynote on day 2, which should be poignant and interesting.  See you there!

I’ve not attended an Intel event before, however I’m interested to see [...] ]]> Next week I’m travelling again and will be in the San Francisco area for the week.  For Tuesday & Wednesday I’m at the Intel IDF (http://www.intel.com/idf/) and for Thursday & Friday I’ll be attending Tech Field Day #8 (http://techfieldday.com/2011/tfd8/).

I’ve not attended an Intel event before, however I’m interested to see their views on Storage and Cloud Computing futures as they have a vested interest in the success of these (and of course many other) technologies.

Currently TFD#8 no presenting companies have yet been announced, however knowing the history of previous events (I attended the very first in November 2009), I’m sure we’ll be seeing some interesting technology.

If you’re at the Intel event, drop me a note (@chrismevans on Twitter) and we can meet up.

Physical servers were usually under-utilized and took time and effort to deploy. These servers also consumed data center space, power and cooling. Virtualization reduced hardware costs, reduced the environmental [...] ]]> There’s no denying that virtualization platforms such as VMware and Hyper-V have revolutionized the way in which computing resources are deployed.

Physical servers were usually under-utilized and took time and effort to deploy. These servers also consumed data center space, power and cooling. Virtualization reduced hardware costs, reduced the environmental requirements by saving on power and cooling and improved the utilization of physical hardware in comparison to dedicated server environments.

Read the rest of this article at Datamation.

HP have been pretty [...] ]]> So there we have it.  A week after Dell announce their intention to acquire 3Par, HP put their cards on the table and trump the Dell bid with an updated offer of $24 a share.  This represents a 1/3 increase over the Dell $18/share offer.

HP have been pretty acquisitive in the storage arena over the last few years, acquiring LeftHand, Ibrix and others.  Why now would they want to acquire 3Par?

• Defensive positioning – 3Par being acquired by any of the major vendors could weaken HP’s position in the mid-to-enterprise market.  EVA is a fading product and unlikely to be the first choice for many organisations when choosing a mid-tier storage array (unless you’re an out and out HP shop of course).  Maybe HP felt they had to protect their existing mid-market position.
• Technology Replacement - As I just mentioned, the EVA looks long in the tooth.  Should HP replace or upgrade it?  Acquiring 3Par could be the answer to that problem – use InServ as the EVA replacement.  Goodbye EVA.  Alternatively, 3Par could be a replacement for the XP range of devices that HP OEM from Hitachi.  HDS are being brought closer to the Hitachi family (and making acquisitions of their own) so perhaps this is another defensive move on HP’s part, in case the Hitachi deal unravels.

Whatever the reason, HP & 3Par would create a large overlapping portfolio of similar products.  I’m not sure where the clear strategy would be, or what the marketing message would say; which product is best for enterprise, mid, SME and so on.

Final thought:  Will Dell rebid at a higher price?  Will anyone else come into the bidding?  Storage is clearly still hot acquisition territory.

In any system, resources are finite.  There is always a limitation to what is available.  However there’s also a truism that [...] ]]> This is a series of posts covering the subject of Storage Management.  Previous posts:

• The Four Pillars of Storage Management
• Four Pillars: Service
• Four Pillars: The Service Catalogue

In any system, resources are finite.  There is always a limitation to what is available.  However there’s also a truism that states if resources are free then they will be consumed at an infinite rate.  So it is with storage.  Someone has to pay for the storage resources that are placed on the floor.  If customers are not charged in some way for their consumption of storage, then they will continue to consume resources ad infinitum.  The solution is to implement chargeback or, to be more precise, billing.

Definition

It’s worth pausing for a moment and discussing the terms Chargeback and Billing.  When computing was first made available as timesharing, customers were billed for their usage of the shared system.  The billing unit may have been time, CPU resources or some combination of metrics that represented utilisation.  Mainframe resources were so expensive that there had to be an efficient charging mechanism.   The concept of billing is something that was intrisically built into the mainframe design and even to this day, resources can be tracked using records produced by SMF (System Management Facility) and reported on through RMF (Resource Measurement Facility).  So billing represented a method of charging for usage that wasn’t directly related to the underlying hardware.

Chargeback implies a different methodology where the direct cost of delivering the service is charged back to the customer.  This can include people costs, but typically hasn’t, only covering the hardware provided itself.  Chargeback has its place, but when looking to develop a service, isn’t as flexible as billing.  All too often, chargeback is tied to a poorly implemented service catalog (or non-existent one).  Whilst the customer may pay for their equipment, there isn’t any flexibility when it comes to hardware replacement as the customer is aware of the technlogy used to deliver their service (and may be unwilling to move to new, untried hardware).  Here are a few additional chargeback/billing combinations that could be implemented:

• No chargeback – IT has a budget and they provide the resources to the business.  When resources are exhausted, the business have to justify or provide additional funds.
• Consumption-based – customers are charged directly for their usage.
• Shared-usage – customers are charged a share of the costs, not directly related to their usage, but perhaps size of business unit.
• Dedicated – customers are charged the whole cost of acquiring the technology.  Ths doesn’t work well for shared environments.
• Service-based - customers are charged for a service provided; this isn’t directly related to the specific technology in use.

Rationale

Whether you are implementing chargeback or billing, there needs to be a good reason for implementing.  Here are a few for consideration.

• To Reduce Costs – If resources appear to be free they will be consumed inefficiently; charging for usage helps controls this.
• Improved Utilisation – Being charged in proportion to your usage makes customers validate whether they really need the storage they are using.
• Improved Efficiency – this goes hand in hand with utilisation, however charging customers for storage can enable tiering to be implemented more efficiently.
• Charging Fairly – there will always be sensible customers and abusers (the broadband market shows us that).
• Manage Demand – It is possible to make charges both time and planning dependent (more on that later).
• Manage Tech Refresh – Abstracting cost and service catalogue from the hardware means new/cheaper/efficient technology can be introduced more easily.

What’s clear from the above points is that chargeback/billing can be used to change customer behaviour; users can be incentivised to be more efficient or to use cheaper technology.  Structured correctly, the overall cost of delivery of storage can include refresh funding, so as old devices are decommissioned, the cost of data migration is part of the overall charge.  I see this as one of the major issues with the way customers pay for their technology; the overall costs in the lifecycle of deployment, operation and refresh simply aren’t considered.

Metrics

What’s the best way to charge?  Here are a few typical metrics:

• Per GB of storage used.
• Per port on the SAN fabric.
• By Tier of storage.
• By contention ratio of storage port (higher cost for fewer hosts on a shared port)
• Charge for replication (both local and remote)
• Charge for deduplication (which may be a lower cost)
• Charge for thin versus thick provisioned LUNs
• Charge for SAN network bandwidth
• Charge for multi-path software
• charge for online backup copies
• charge for offline backup copies

Whatever metrics are used, the key intent is to charge for customer use of a service.  This needs to be abstract enough to be disconnected from technology, so charging for fibre channel ports may be too prescriptive; the cost may be described as “to be connected to the SAN” in general, providing a blended charge that would cover iSCSI, Fibre Channel or FCoE connectivity.

Implementing a Chargeback Process

As part of the implementation process, it’s worth considering having billing/chargeback principles established.  These can be provided to the customer.  Here are some examples:

• The charging model will be based on resource consumption of each user independently (e.g. user changing their utilisation doesn’t affect another user)
• Charging costs will be reviewed and changed on an annual/bi-annual/quarterly basis from 1 Jan 200x
• Charging will be based on storage in use on 28th day of each month
• Charging will/will not be based on utilisation (rather than allocation)
• Charging will be attributed at the host/server/LUN/file level
• A target of 100% cost recovery is the target goal
• Charging may result in an IT surplus/deficit from year to year, but will be a non-profit business
• Billing charges will be based on the published “Storage Catalogue”
• Users of equipment classed as legacy will be notified 6 months in advance of technology acquiring legacy status
• IT/Storage Team will strive to deliver price stability and/or reductions year-on-year
• Chargeback will be implemented as evolution rather than revolution

The internal cost of delivery of storage will include:

• Hardware and software costs
• Additional feature licences
• Power/cooling/space (environmental costs)
• People costs
• Training
• Network costs
• DR costs

There may be more, depending on how your technology is delivered (for instance managed data centres), but what’s essential is to baseline what it takes to deliver the service.  Quite simply the process would be:

1. Identify service cost components (as above)
2. Identify consumption metrics (service charging units)
3. Measure use
4. Model costs based on consumption metrics
5. Bill customers.

Other considerations, which I’ll save for future posts are:

• Standards – how they are important to chargeback
• Measuring tools
• Measurement interval
• Incentivising customer behaviour in favour of technology refresh
• Outsourcing some components
• Determining the customer
• Forecasting/Capacity Planning

There’s lots more to come, feedback on the article so far is very welcome.

I worked for Lehman in the UK for a short period about 4 years ago. If their banking operations was run anything like IT, then there’s no [...] ]]> The latest casualty of the credit crunch is Lehman Brothers who have filed for Chapter 11 Bankrupcy protection. See the report here from the BBC.

I worked for Lehman in the UK for a short period about 4 years ago. If their banking operations was run anything like IT, then there’s no surprise they are out of business today.

I joined as a consultant after the entire storage team left for other jobs and within 6 months moved on myself after the outsourcing Sword of Damocles hovered over the team. The storage operation in place was fundamentally flawed, however based on the attention Lehman’s will be getting, this may not be the best time for me to expand on those issues in writing.