The Storage Architect » Universal Volume Manager

Enterprise Computing: Data Migration Strategies – Part V

Chris M Evans — Wed, 22 Jul 2009 13:00:54 +0000

This is the final post in a series on Enterprise Data Migration Strategies. Previous posts:

Enterprise Computing: Data Migration Strategies – Part I

Enterprise Computing: Data Migration Strategies – Part II

Enterprise Computing: Data Migration Strategies Part III

Enterprise Computing: Data Migration Strategies – Part IV

Previously we’ve discussed how to plan, structure and organise migrations. In this post, I’ll touch on some of the tools which may be used to perform migration work.

One Size Does Not Fit All

First of all, its worth pointing out that no single solution fits all needs; migration methods are varied and the specific configuration in place demands the best solution at the time. Therefore it pays to have an arsenal of tools at your disposal and know how you’d use each one.

Array-Based Migration

Storage arrays already have many tools for performing migrations. These exist today for business purposes; remote replication to another array; local replication within an array using clones and snapshots. The benefit of using in-array technology is the migration work is taken away from the host and potentially can be executed within minimal customer interaction. On the negative side, most replication technologies which move data between arrays are product specific – i.e. SRDF on EMC DMX arrays isn’t compatible with HDS’ TrueCopy. This shouldn’t be a surprise because they are proprietary technologies which gain their advantage by being specifically coded and optimised to the storage platform itself. There are however tools like EMC’s Open Replicator which can move data between vendor/family technology. Open Replicator does have restrictions though – depending on the type/direction of replication, incremental copying isn’t available and requires a full copy sync to complete, potentially removing the benefit of using the tool altogether.

Virtualised Migration

Sitting slightly higher up the “storage stack”, it is possible to do migrations using a virtualisation technology sitting above (or integrated with) the storage array. For example, IBM’s SVC can be used to manage data migrations and sits above all storage arrays; HDS’ USP (equivalent to HP XP models) has a facility called Universal Volume Manager (UVM) which can perform the same work and is built into the array. Incipient have a solution called INSP (Incipient Network Storage Platform). If not already deployed, these tools will need an outage to be installed in the data path. Both impact the World Wide Name (WWN) the host sees, so host changes may also be necessary, depending on operating system. The benefit of these technologies, once installed, is that they allow data to be moved dynamically “under the covers” without involvement of the customer or work on the host server. As with all technologies, there are restrictions under certain circumstances and you should check with the product vendor for those. It may well be that you want to move the virtualisation tool at the end of the migration so another outage may also be required.

Fabric Migration

Moving slighly higher, we have the ability to perform data migrations in the storage fabric (SAN) itself. Example products include Brocade’s Data Migration Manager (DMM) and EMC’s InVista. Storage migration in-fabric requires the deployment of hardware in a SAN switch that intercepts I/O and redirects a second copy to another device. Potentially these devices can be installed in the data path without distruption but will require an outage to cut over to the new target volumes.

Host Migration

Finally at the top of the stack we have host-based migrations. Even at this level there are still a number of choices. If a Logical Volume Manager is installed (e.g. Veritas Foundation Suite/Volume Manager), then migrations can be performed using this software without host disruption. This is often a good choice of tool if the target devices are in a different array, if outages can’t be taken or if the LUNs are being re-organised or restructured. Unfortunately this also means having either host-access given to the storage teams (plus O/S knowledge to complete the work) or requiring the platform teams to perform the migration work. Both of these options may be a problem in certain organisational structures. One word of warning using LVMs – if LUNs are being replaced by using “evacuate” functionality (where a LUN at a time is swapped with another) then a potential data integrity problem exists, especially if the LUNs are also replicated. The risk occurs because data spans two arrays and if remotely replicated, then writes at the DR site might not be in integrity timestamp order. Failure in either array can result in an outage.

For mainframe customers, there’s the fantastic TDMF (also available in an Open Systems version)

If LVMs are not available, then good old-fashioned data copying is the order of the day. There are many tools to do this, too numerous to mention here, but be aware, that this method is likely to mean protracted downtime as storage shouldn’t be active and be accessed whilst it is being copied. It is also possible to migrate data within an application, again, there are too many options to mention here.

Hopefully this article provides a flavour of the migration tools out there. Please add comments or ping me if you’ve any specific tools you would like me to mention and I’ll add them on as a separate page.

Enterprise Computing: HDS Switches On Virtualisation For Free

Chris M Evans — Wed, 22 Apr 2009 06:16:50 +0000

There’s no doubting HDS‘ Universal Volume Manager (UVM), aka external storage virtualisation is a cool product. I’ve used it many times – it does the job. However, the main drawback to using the product for me was always cost (I mentioned this only a few weeks ago on this post). Well, now that’s changed; until the end of this year, HDS are offering UVM for free. See the announcement here. HDS are calling it their “Switch It On” campaign.

OK, free isn’t quite free – there are a few caveats. Customers have to pay for maintenance, but other than that, there’s no charge for using UVM on current USP-V deployments. In addition, customers can “super-size” the offer and also get Hitachi Tiered Storage Manager (HTSM), Hitachi In-System Replication and limited access to Hitachi Dynamic Provisioning (HDP) all for free.

It is unlikely HDS are doing this out of some altruistic concern for their customers. Clearly they see benefits in driving more business by offering these licences at no cost. So how can customers benefit?

Traditional Approach

Traditionally, UVM is seen as a tool to manage the migration process or get data into a USP and there’s no question that UVM can be used for these purposes. Typically, here are some of the costs associated with migration:

Staff Resource Costs – migrations take time and effort to plan. The longer they take, the more cost is incurred with change control, planning, co-ordination with other teams and on actually performing the migration, which is typically performed out of hours.
Software Costs – migrations may require specific software products or tools (e.g UVM or TDMF).

Here are some of the constraints on migration work:

Limited downtime on servers.
Data to be migrated out of hours or within certain time frames.
No host tools (like volume managers) to perform migrations.
Data migrations are required to disparate locations.
Data integrity and synchronicity must be maintained (e.g. replicated data must be consistent if an outage occurred during the migration process.

The tradeoff with UVM has always been the cost of the UVM licence compared to the savings which could be made on the above issues by using the product. For example, UVM could be used to virtualise an entire array (or arrays) behind a USP-V in a single weekend. Consecutive weekends can then be used to migrate into the USP. This controlled approach (a) enables the storage managers to perform the migration, freeing up the host teams (b) allows the storage managers to load balance migrations into the USP-V (c) monitor performance as migrations occur (d) enable target servers to be immediately upgraded to new code/driver levels which may not have been supported previously on the older arrays.

Lateral Thinking

Getting UVM for free enables some more lateral thinking:

Example 1: Migrate out of the USP. There are lots of examples of moving data into a USP, but what about moving it out? If you’ve purchased a USP and it’s fully populated with disk and full, how do you save money? Well, one option is to get a UVM licence and migrate non-critical (and perhaps non-production) data to an externalised array at a lower cost. Previously this could only be commercially viable if the reduced cost of the external array covered the cost of the UVM licence. Now there’s no need to worry.
Example 2: Migrate in and De-Dupe. With UVM and HDP, external storage arrays can be migrated into the USP and placed on thin provisioned volumes. HDP Zero Page Reclaim removes the “empty” blocks of data during this process. For certain data profiles, this could provide signficant savings.
Example 3: A Free Datacentre Migration Tool. Imagine you’ve got to move data on arrays not currently replicated – but that data needs to reside in another location. If a pair of USPs are available, why not virtualise the source and target arrays and use the USP to move the data between the two sites? This could save purchasing replication licences for an array or resolve a problem where replication isn’t possible from certain hardware.
Example 4: Implement a 3DC Solution. How about using the USP-V as a three datacentre solution? Here’s how it works; two USP-Vs are used to replicate data synchronously between two local sites. The target devices are actually virtualised across a fabric to a third site using UVM. In this way, data can be moved to a third site without requiring a third copy of data.

With free software, the options are only limited by imagination. If HDS are prepared to offer software for free then there’s no excuse not to use it.

Enterprise Computing: Using USP for Migrations

Chris M Evans — Sat, 24 Jan 2009 17:01:00 +0000

Thanks to Hu Yoshida for the reference to a previous post of mine which mentioned using virtualisation (USP, SVC, take your pick) for performing data migrations. As Hu rightly points out, the USP, USP-V, NSC55 and USP-VM can all be used to virtualise other arrays and migrate data into the USP as part of a new deployment. However nothing is ever as straightforward as it seems. This post will discuss the considerations in using a USP to virtualise and migrate data into a USP array from external sources.

Background

The Universal Volume Manager (UVM) feature on the USP enables LUN virtualisation. To access external storage, storage ports on the USP are configured as “External” and connected either directly or through a fabric to the external storage. See the first diagram as an example of how this works.

As far as the external storage is concerned, the USP is a Windows host and the settings on the array should match this. Within Storage Navigator, each externally presented LUN appears as a RAID group. This RAID group can then be presented as a single LUN or if required, carved up into multiple individual LUNs.

The ability to subdivide external storage isn’t often mentioned by HDS; it’s usually assumed that external storage will be passed through the USP on a 1:1 basis and if the external storage is to be detached in the future then this is essential. However if a configuration is being built from scratch then external storage could be presented as larger LUNs and subdivided within the USP. This is highlighted in the second diagram.

At this point, external storage is being passed through the USP but the data still resides on the external array. The next step is to move the data onto LUNs within the USP itself. Here’s the tricky part. The target LUNs in the USP need to be exactly the same size as the source LUNs on the external array. What’s more, they need to be the same size as the way the USP views them – which is *not* necessarily the same as the size on the external storage itself. This LUN size issue occurs because of the way the USP represents storage in units of tracks. From experience, the best way to solve this problem was to actually present the LUN to the USP and see what size the LUN appears as. When I first used UVM, HDS were unable to provide a definitive method to calculate the size a LUN would appear within Storage Navigator.

The benefits of virtualisation for migration can fall down at this point. If the source array is particularly badly laid out, the target array will retain the multiple LUN sizes. In addition, a lot of planning needs to be performed to ensure the migration of the LUNs into the USP doesn’t suffer from performance issues.

Data is migrated into the USP using Volume Migration, ShadowImage (or TSM). This clones the source LUN within the USP to a LUN on an internal RAID group. At this point, depending on the migration tool it may be necessary to stop the host to remap to the new LUNs. This completes the migration process. See the additional diagrams, which conceptualise migration with TSM.

Now, this example is simple; imagine the complexities if the source array is replicated. Replication has to be broken, potentially requiring an outage for the host. Replication needs to be re-established within the USP but data has to be fully replicated to the remote location before the host data can be confirmed as consistent for recovery. This process could take some time.

Summary

In summary, here are the points that must be considered when using USP virtualisation for migration:

Configuring the external array to the USP requires licensing Universal Volume Manager.
UVM is not free!
Storage ports on the USP have to be reserved for connecting to the external storage.
LUN sizes from the source array have to be retained.
LUN sizes aren’t guaranteed to be exactly the same as the source array.
Once “externalised” LUNs are replicated into the USP using ShadowImage/TSM/VM.
A host outage may be required to re-zone and present the new LUNs to the host.
If the source array is replicated, this adds additional complication.

I’ll be writing this blog up as a white paper on my consulting company’s website at www.brookend.com. Once it’s up, I’ll post a link on the blog. If anyone needs help with this kind of migration, then please let me know!