• EMC Symmetrix/DMX – Physical disks are carved into smaller segments called hypers. These are then recombined into LUNs which then might be recombined into composite devices (metas) and replicated, cloned or snapped. The hypers that make up a LUN can come from anywhere within an array and can be moved around at will by a tool designed to improve performance, completely ruining your original well-planned configuration. Combining hypers give you RAID, which wasn’t RAID before and was something called mirrors but is now, and is even RAID-6! Devices have personalities which survive their presentation or removal from a port. A device can have multiple personalities at the same time. LUNs use a nice numbering system based on hex – but don’t expect them to number nicely if you destroy and create devices. Bit settings (flags) are used to ensure host SCSI commands work correctly.
• HDS USP/HP XP – Physical disks are grouped into RAID groups from which LUNs are carved. Until recently you couldn’t span RAID groups easily (unless you were combining some free space in each RAID group). Devices don’t have a personality until they’re presented to a host on a port, but they can have multiple personalities. HDS use a form of punishment known as CCI for anyone foolish enough to think they had made their arrays easy to manage. LUNs are numbered using a relic of the mainframe and yes, you can move things around to balance performance, but don’t think you can do it unless there are spare LUNs (sorry LDEVs) around. Different host types are supported by a setting on a host group which lets you confuse the hell out of every one by telling them their LUN numbers are all the same but unique. Oh, and the storage the user sees doesn’t actually have to be in the array itself.
• HP EVA – Phew! Physical disks are managed in groups (which it’s recommended to only have one of, but you can have more if you really must) but they don’t use RAID at the group level because that would be too easy. Instead disks are grouped into Redundancy Storage Sets, which reduce the *risk* of disk failures but don’t protect directly against them. LUNs are created only when they need to be presented to a host and they don’t have simple LUN numbers, but rather 32 digit UUIDs. RAID protection is done at the LUN level, making it more difficult to conceptualise than either of the previous two examples.
• Pillar Axiom – now we’re getting really abstract. With Axiom, you can tier data on different levels of performance, but wait for it – they will be on the same drive, but utilising different parts of the same spindle! Argh! Enough!

Clearly every vendor wants to differentiate their product so you’ll buy from them and not the competition. In some respects they *have* to differentiate otherwise all the vendors would spend their time in litigation with each other over patent copyright! (wait a minute, they already are). So SMI-S or any other standard is going to have a near impossible time creating a single reference point. Add to the mix the need to retain some competitive advantage (a bit like Microsoft holding back the really useful API calls in Windows) and to sell their own management tools and you can see why SMI-S will be at best a watered down generic interface.

So why bother. There’s no benefit. Every vendor will give lip service to the standard and implement just what they can get away with.

The question is, what would replace it? There’s no doubt something is needed. Most SRM tools are either overbloated, poorly implemented, expensive, or plainly don’t work so some light touch software is a must.

I think the interim solution is to get vendors to conform to a standard API format, for example XML via an IP connection to the array. Then leave it to the vendor how to code up commands for querying or modifying the array. At least the access method would be consistent. We don’t even see that today. All we need now is an acronym. How about Common Resource Access Protocol?

The $5000/TB figure they were quoting was an average they had seen in the industry for certain vendors’ professional services time to come in and perform the migration work on behalf of the customer. Incipient’s take was that they could provide their appliance/software expertise to provide the same service but at a significantly reduced cost (I won’t quote specific numbers here, but the number quoted was much lower than the equivalent cost from “a vendor”). So, I guess with clarification, it is more clear that Incipient were comparing the vendor costs versus their product costs and not including any internal customer costs (project management, preparation work etc) in the calculation. This seems a more appropriate comparison in my opinion.

Getting back to the vendor discussion, there’s a real issue here. If vendor X wants to sell you their latest technology, they need to accept and take the hit on helping with migration to their new array. This should be even more so where the vendor doesn’t change as this should be a “no brainer” and built into the technology.

In a world where hardware is becoming a commodity, one differentiator will be the vendor who can minimise the effort/cost and impact of migrating from one technology to another. Until then, products like SVC and those from Incipient will continue to have a market position – oh and humble consultants like yours truly!

The UK on the other hand must have one of the most complex ticketing systems, especially around London. As an example, if I travel into London from where I live and want to return home between 4pm and 7pm then I can’t buy a cheap day return. Presumably that’s because they can fleece travellers who don’t realise this rule exists. However if I am already in London and want to travel out, I can buy a cheap day single and travel on it between 4pm and 7pm! Even the people selling the tickets think it is crazy. I could give you dozen’s of other similar examples, but life’s too short.

So it is with storage. Keep it simple. Take tiering as an example. You could spend days and weeks developing the most finely detailed tiering strategy but in reality you will find most data will sit on a small number of tiers, the bulk of it being in the middle range. Developing complex tiering structures, just like complicated train pricing structures just leads to confusion and in the end additional cost. All that’s needed is a simple strategy with most of the data on cost efficient storage.

Remember – simple is good.

• An engineer was testing the Halon system in the newly completed computer room extension at my first site. Unfortunately he’d forgotten to turn the key to “test” before pressing the fire button and let off the Halon in the whole of the datacentre with both the equipment up and running and operators in the room mounting tapes. Needless to say, they were out of there like a rat up a drainpipe!
• During a recent delivery of storage arrays; one array literally fell of the back of the lorry. It had to be shipped back for repair…
• An array installation I managed in one site was mis-cabled by both the electricians and the vendor. When it was powered up, it exploded…
• On a delivery of equipment, the vendor arrived at the loading bay at the datacentre. As the loading bay door was opened, it jammed and broke, just too low for the arrays being delivered to be pushed under the door. The vendor had to return the following day after the broken door had been repaired.
• A tape drive on a StorageTek library I worked on took 12 hours and around 6 staff to complete. Half way through the upgrade, we took a go/no go point and checked both the MVS and VM installations to ensure the new drives worked. The MVS connected drives were fine; the VM drives had a “minor problem”, so we proceeded, in anticipation of resolving the VM problem. The following day we discovered the VM problem was not correctable and had to purchase additional drives at considerable cost.
• After loaning out some disk space to a “temporary” project, we had a hardware failure 3 months later. It turned out that the team had forgotten to ask for backups for their data and 3 months of the work of a dozen people was lost.

Fortunately, most of the above were not life threatening (except the first, which I was not involved in directly). However one of these problems did result in data loss (albeit on a development environment). It shows how many times the unexpected and unplanned can happen and mess up the best laid plans.

Care to share any of your stories?

Compare this to our trader friends. Jerome Kerviel allegedly managed to gamble €50bn, more than the assets of his bank, Societe Generale. I wonder how many change records he had to raise in order to get access to that level of funds?