I talked about Incipient last [...] ]]> Earlier this month, Texas Memory Systems announced they had acquired the intellectual assets of Incipient, a company that produced SAN virtualisation hardware and software.  With Incipient gone, EMC hardly bothering to mention Invista, what is the future of SAN LUN virtualisation? 

I talked about Incipient last year, here and here when discussing the costs of performing migrations.  As I said at the time, I couldn’t see how much of a saving deploying their iNSP would bring to the burdensome migration work we all have to manage on an ongoing basis.  So there’s got to be a more compelling benefit out there for using virtualisation products.  If there is, then what is it?

Excluding the defunct Invista, that leaves Hitachi with Universal Volume Manager (UVM) and IBM with SAN Volume Controller (SVC) still in the market place.    From experience, I know UVM is a great product and surprise, I’ve commented on that recently too especially herewhere I reference the fact that Hitachi are offering UVM for free.  Clearly, the drawback to UVM is that it is integrated into the array itself.  When the NSC55 first came out, I heard rumours that it may be a diskless virtualisation “head” and although it can be deployed in that way, it isn’t sold as that.  If Hitachi decided offer the USP VM or its successor as a diskless virtualisation controller, it would put them squarely in competition with SVC from IBM.

Earlier this year I was fortunate to have an invitation to meet Barry Whyte, “Master Inventor” and Performance Architect on the SVC product.  You can find Barry’s blog hereif you’re already not subscribed to it.  I highly recommend it especially for understanding the in’s and out’s of the SVC itself.  During my trip I got to see some of the hardware used to do interoperability testing of SVC – with storage it virtualises as well as servers it connects to.  It’s by no means a trivial task; there are 80 people in Hursley alone, working on development and testing of the product as well as a further 64 scattered around the globe.  Obviously virtualising storage is a complex business and requires huge amounts of testing.  I’d go as far as suggesting that the testing takes way more cycles than writing the code itself.

What’s all this got to do with the future of virtualisation?  Well, I think it highlights what a complex process it is.  Even though standards for interoperability exist, IBM (and presumably Hitachi, EMC and at one time Incipient) have to deal with complex interoperability issues and interleave that with additional features and functionality whilst guaranteeing data integrity.  The slide taken from an SVC presentation deck gives you an idea of what’s involved.  Thanks to Barry for permission to reproduce this.

Both Hitachi and IBM have been successful with a virtualisation product that doesn’t sit within the SAN fabric itself.  This seems to me to be counter-intuitive as I’ve always thought the fabric was the right place for virtualisation.  After all, every I/O leaving a host hits the fabric first and this naturally becomes the best place to route the I/O to its final destination, whether or not that is a “real” LUN or one created from a virtualisation product. 

Perhaps SAN fabric virtualisation was simply too complex and costly to deploy.  After all, recent history has told us that paying for a fabric-based virtualisation product is a non-starter otherwise we’d see more Invista and iNSP.  Perhaps fabric-based virtualisation didn’t provide the feature set that mature IT organisations required from the technology.  Either way, virtualisation in the fabric needs a rethink.  Maybe FCoE provides/provided that opportunity?

Enterprise Computing: Data Migration Strategies – Part I

Enterprise Computing: Data Migration Strategies – Part II

Enterprise Computing: Data Migration Strategies [...] ]]> This is the final post in a series on Enterprise Data Migration Strategies.  Previous posts:

Enterprise Computing: Data Migration Strategies – Part I

Enterprise Computing: Data Migration Strategies – Part II

Enterprise Computing: Data Migration Strategies Part III

Enterprise Computing: Data Migration Strategies – Part IV

Previously we’ve discussed how to plan, structure and organise migrations.  In this post, I’ll touch on some of the tools which may be used to perform migration work.

One Size Does Not Fit All

First of all, its worth pointing out that no single solution fits all needs; migration methods are varied and the specific configuration in place demands the best solution at the time.  Therefore it pays to have an arsenal of tools at your disposal and know how you’d use each one.

Array-Based Migration

Storage arrays already have many tools for performing migrations.  These exist today for business purposes; remote replication to another array; local replication within an array using clones and snapshots.  The benefit of using in-array technology is the migration work is taken away from the host and potentially can be executed within minimal customer interaction.  On the negative side, most replication technologies which move data between arrays are product specific – i.e. SRDF on EMC DMX arrays isn’t compatible with HDS’ TrueCopy.  This shouldn’t be a surprise because they are proprietary technologies which gain their advantage by being specifically coded and optimised to the storage platform itself.  There are however tools like EMC’s Open Replicator which can move data between vendor/family technology.  Open Replicator does have restrictions though – depending on the type/direction of replication, incremental copying isn’t available and requires a full copy sync to complete, potentially removing the benefit of using the tool altogether.

Virtualised Migration

Sitting slightly higher up the “storage stack”, it is possible to do migrations using a virtualisation technology sitting above (or integrated with) the storage array.  For example, IBM’s SVC can be used to manage data migrations and sits above all storage arrays; HDS’ USP (equivalent to HP XP models) has a facility called Universal Volume Manager (UVM) which can perform the same work and is built into the array.  Incipient have a solution called INSP (Incipient Network Storage Platform).  If not already deployed, these tools will need an outage to be installed in the data path.  Both impact the World Wide Name (WWN) the host sees, so host changes may also be necessary, depending on operating system.  The benefit of these technologies, once installed, is that they allow data to be moved dynamically “under the covers” without involvement of the customer or work on the host server.  As with all technologies, there are restrictions under certain circumstances and you should check with the product vendor for those.  It may well be that you want to move the virtualisation tool at the end of the migration so another outage may also be required.

Fabric Migration

Moving slighly higher, we have the ability to perform data migrations in the storage fabric (SAN) itself.  Example products include Brocade’s Data Migration Manager (DMM) and EMC’s InVista.  Storage migration in-fabric requires the deployment of hardware in a SAN switch that intercepts I/O and redirects a second copy to another device.  Potentially these devices can be installed in the data path without distruption but will require an outage to cut over to the new target volumes.

Host Migration

Finally at the top of the stack we have host-based migrations.  Even at this level there are still a number of choices.  If a Logical Volume Manager is installed (e.g. Veritas Foundation Suite/Volume Manager), then migrations can be performed using this software without host disruption.  This is often a good choice of tool if the target devices are in a different array, if outages can’t be taken or if the LUNs are being re-organised or restructured.  Unfortunately this also means having either host-access given to the storage teams (plus O/S knowledge to complete the work) or requiring the platform teams to perform the migration work.  Both of these options may be a problem in certain organisational structures.  One word of warning using LVMs – if LUNs are being replaced by using “evacuate” functionality (where a LUN at a time is swapped with another) then a potential data integrity problem exists, especially if the LUNs are also replicated.  The risk occurs because data spans two arrays and if remotely replicated, then writes at the DR site might not be in integrity timestamp order.  Failure in either array can result in an outage.

For mainframe customers, there’s the fantastic TDMF (also available in an Open Systems version)

If LVMs are not available, then good old-fashioned data copying is the order of the day.  There are many tools to do this, too numerous to mention here, but be aware, that this method is likely to mean protracted downtime as storage shouldn’t be active and be accessed whilst it is being copied.  It is also possible to migrate data within an application, again, there are too many options to mention here.

Hopefully this article provides a flavour of the migration tools out there.  Please add comments or ping me if you’ve any specific tools you would like me to mention and I’ll add them on as a separate page.

I had a quick check on EMC’s site and the last press release for the [...] ]]> Remember when EMC offered a fabric-based storage virtualisation product called InVista? Apparently they still do, as a customer of mine mentioned recently. Unfortunately they had trouble getting any sort of decent reference site for the product and weren’t that impressed.

I had a quick check on EMC’s site and the last press release for the product I could find was dated December 8th 2007. Compare that to the news about DMX, Clariion and probably almost every other product put out by EMC and InVista doesn’t get much press. In fact general opinion perceives InVista as EMC’s problem child, finding trouble gaining acceptance with storage managers at large. Why?

To be honest, I’m not sure why. The *concept* of fabric-based virtualisation seems appealing and once the appliance has been integrated into a SAN, data migration should be simple. Perhaps there’s a lack of trust in the scalability or performance, perhaps a lack of trust that the product actually works, perhaps a nervousness in placing yet another layer into the infrastructure which, if it fails, could be massively disruptive to normal storage operations.

That leads to the question – what should EMC do? I doubt they will drop the product, as that will give competitors (especially IBM & SVC) too much of a marketing opportunity. Maybe they will acquire a competitor (like Incipient) and slowly transform InVista into a new product. I don’t have a crystal ball to answer that one.

I’ll close with one thought – are there any other products with the word “Vista” out there that are also equally dismissed and derided?

Now, if you want to know about [...] ]]> I’ve not paid much attention to Incipient (their news page doesn’t provide an RSS feed, so there’s no chance of me seeing their press releases easily), but my attention was recently drawn to a recent release relating to their iADM and iNSP products (catchy names, those).

Now, if you want to know about their products, have a look at their website for yourself. Rather, my interest was sparked by a claim in their press release, quoted below:

The High Cost of Today’s Data Migration

Industry estimates and field data captured by Incipient indicate that SAN storage is growing at 40 – 60 percent annually and 25 percent of data under management is moved annually at an average cost of $5,000 per terabyte. Based on these estimates, a data center with one petabyte of storage under management today spends $1.25 million annually on data migration operations. Two years later, the data center is likely to grow to nearly two petabytes increasing the annual data migration cost to nearly $2.5 million.

Source: Incipient Press Release 11 June 2008

So the estimate is $5000 per TB of data movement and 25% of data being moved each year. I can understand the latter; it’s simple logic that if you have a 3-4 year lifecycle on technology then on average 25% of your estate will be being refreshed each year (although that figure is slightly distorted by the fact that you’re also deploying an additional 40-60% each year). Now, how to get to a $5000 per TB calculation…

Excluding new storage acquisition, network bandwidth, etc, I’d assume that the majority of migration costs will be people time. That would include planning and execution of migrations. In environments of 1PB or more, I could (almost) bet my house on the fact that there will be a significant amount of the storage infrastructure which is (a) not understood (b) badly deployed (c) backlevel amongst many other issues. $5000/TB would therefore seem quite reasonable, based on the amount of work needed to refresh. The only problem, though, is that a majority of the manpower cannot be solved by software alone. This will include documenting the environment, bringing server O/S, firmware and drivers up to date, negotiating with customers for data migrations, migration schedule planning, clearing up wastage, new server hardware and so on.

It would be an interesting exercise to determine what percentage of the $5000/TB cost is actually attributable to data movement work (i.e. having someone sitting at a screen issuing data replication commands). I suspect it is quite low. From experience, I’ve been able to move large volumes of data in quite short timespans. In fact assuming sensible preparation and planning, most of the time doing migrations is sitting around (previous employers disregard this statement).

So how much money would Incipient save? My bet is not much.