Allegedly, Barclays now use USP-V arrays as their [...] ]]> It’s been reported in a few places that yesterday Barclays (UK bank) suffered an issue with a “disc array” (presumably they mean disk array) that took out their ATM and online banking systems.  See the comments here and here.

Allegedly, Barclays now use USP-V arrays as their back-end storage devices, so presumably HDS USP-Vs were involved in yesterday’s problems.  Systems seemed to have been down for a number of hours before normal service was resumed.

The first thing to say is that “stuff” happens.  Hardware fails – arrays fail and it’s the same for all vendors.  No vendor can ever claim that their hardware doesn’t fail once in a while.  We all know that RAID is not infallible; in fact, it isn’t even necessary to have a hardware failure to experience service outage as many problems are caused by human error.  

What surprises me with this story is the time Barclays appeared to take to recover from the original incident.  If a storage array is supporting a number of critical applications including online banking and ATMs, then surely a high degree of resilience has been built in that caters for more than just simple hardware failures?  Surely the data and servers supporting ATMs and the web are replicated (in real time) with automated clustered failover or similar technology?

We shouldn’t be focusing here on the technology that failed.  We should be focusing on the process, design and support of the environment that wasn’t able to manage the hardware failure and “re-route” around the problem.  

One other thought.  I wonder if this problem would have been avoided with a bit of Hitachi HAM?

What is on offer is the ability to cluster USPs – a feature called Hitachi High Availability Manager.  [...] ]]> So HDS’s announcement has turned out to be a complete disappointment.  What it’s not:

• It’s not new hardware.
• It’s not providing more physical capacity.
• It’s not providing dynamic tiering.
• It’s not providing enhanced replication technology.

What is on offer is the ability to cluster USPs – a feature called Hitachi High Availability Manager.  By cluster, this means connect two USP arrays together and have them work in an active-active configuration, with data replicated in either direction.  This new feature seems to be answering only one problem - how do I get off my USP?

Back in 2004 (I think it was), when I first sat with HDS and had a presentation on USP, the key question (especially with virtualisation) was how to cope with the fact that a single USP is the SPOF (Single Point of Failure).  People in the room viewed the USP like a network switch - subject to failure, requiring upgrade and so on.  HDS was at pains to say that clustering USPs simply wasn’t necessary as the hardware was fully resilient.  In fact, HDS have been at pains since then to offer 100% availability of data with the USP.  So why is clustering a USP so necessary?  How can we have higher than 100% availability?  Let’s not forget, the level of availability of any system is based on the availability of the least resilient component, so if we have a USP (with 100.00% availability) virtualising external storage (with 99.9% availability), the weak point is the external storage.  Clustering USP’s doesn’t improve this and never will.  All HDS have answered with this offering is the migration issue.  This isn’t a new feature. 

Here are some questions that arise from the presentation and which weren’t answered:

• How far apart can my USP cluster arrays be?
• What’s the impact on latency?
• How is data integrity maintained?
• Does clustering also support TrueCopy, ShadowImage and COW Snapshots?
• Does clustering change my array World Wide Names, and if so, how?
• Can cluster arrays be at different microcode levels?
• Can clusters be TrueCopy secondary devices, if so what replication links are required?
• Do need specific multipathing software?

So, you may be asking why the odd title of this post.  Have a look at here.  It’s what the dolphins said before they left earth.  Time to say goodbye to the USP-V as a player in the Enterprise array space.