HDS/Hitachi I believe were an investor in BlueArc, now they’ve acquired them.  I think this is a good move for both companies for a number of reasons.  Firstly, it marks the continuing change in HDS’ strategy, which now includes acquiring companies, albeit companies that they have an existing relationship with.  This approach may seem cautious but perhaps it’s a good thing to take time to understand the people, technology and culture, although HDS could have made their move earlier.  Second, I believe BlueArc suffers in a market where (large enterprise) customers expect longevity.  Committing to a new platform that suddenly has little or no development could be a costly mistake.  There’s also the issue of global support and maintenance to consider too.  Being part of Hitachi will give customers more reassurance.

Some time ago, I put the following diagram together as part of work for a customer.  it was meant to show how the Hitachi technology can operate on a number of layers, from the physical hardware upwards.  Although I have no specific inside knowledge, I expect Hitachi will converge the physical layer into a single device at some stage.  Chris Mellor’s article hinted at this (although I expect this slide is no longer accurate) and developments in their technology imply this too.  The AMS2XXX range certainly became more USP-like in its last release.

Operating a single hardware platform makes sense as HDS can abstract the hardware from the logical view using tools like HTSM and UVM.  Presentation is then through native FC/FCoE or using gateway products to provide NAS and object connectivity.  VSP represents a much more scalable architecture than the USP was, one of the key features being the abstraction of the front and back-end processors from the physical interfaces.  This scalability easily supports multi-protocol shared environments.

Other vendors have gone down the route of doing scale-out through multiple interconnected nodes.  I think this is a good approach and it could be argued that Hitachi have taken another way and gone for single monolithic arrays.  I don’t believe this is the case as the VSP is already a cluster of two nodes and from the PCIe backplane architecture can presumably scale to more nodes as required.  They are however more closely coupled than other implementations.  If Hitachi delivers on the potential promise of HAM, then clustering/nodes can be implemented and the single monolithic array isn’t an issue.  Now that Hitachi own BlueArc, they can integrate the functionality to ensure any NAS presentation can be mapped back to any storage array, with seamless migration.

So in summary, the future looks good, I’m looking forward to seeing how Hitachi capitalise on their new investment.

Disclaimer: I’ve been engaged with HDS as a blogger, including trips to Japan and the US.  I have worked for HDS UK delivering consultancy to their clients.

http://www.theregister.co.uk/2011/03/29/hds_storage_virtualisation_podcast/

Unfortunately we still didn’t have the Hu v Claus benchpress competiton, but hey….

In this picture are:

• Greg Knieriemen – @knieriemen
• Hu Yoshida
• Claus Mikklesen – @yoclaus
• Nigel Poulton – @nigelpoulton
• Devang Panchigar – @storagenerve

and of course me.

Click here to view the embedded video.

• The Rise of 3Par – HP used the CI event to place 3Par firmly at the front of their storage strategy.  In fact so much so, that the p9500 release seemed overshadowed as a result.  I have more to talk about on this later, but we can be assured, we will see more of the 3Par product  and the 3par team over time.  HPs footnote to the event described 3Par as their storage strategy for the next 10 years.  That’s a big bet.
• IBM Releases New Storage Platform - This week, IBM launched their new V7000 platform, also curiously named Storwize, although it contains no technology from their Storwize acquisition.  The V7000 is clearly based on SVC and borrows technology (such as the management interface from XIV) from other IBM products.  Unfortunately I was unable to attend the launch event, but hope to have some comment after I review the launch material.
• IBM Acquired Netezza - IBM increase their portfolio of technologies with another acquisition, but where will they fit into the portfolio?  Are we going to see new development focused on the V7000 or does XIV still have a future?
• Oracle Announce Exalogic – Yes, Larry moves further forward in his plans for world domination by releasing his own unified computing stack, based on Oracle, Sun and Exadata technology.  At the outset this seems to me like a strapping together of disjoint technologies with the hope that it will play well against VCE (Cisco, VMware, EMC) and HP’s Converged Infrastructure offerings.  Management software for orchestration is of course, the most essential thing here – something I will check out when I have the time.
• Mark Hurd Joins Oracle – speaking of Oracle, HP’s disgraced CEO, Mark Hurd moves to Oracle (as reported here and in many other places).  You have to feel sorry for the previous guy who got pushed out; is anyone in charge of Oracle other than Crazy Larry?
• STEC Release New Ultra Low-Latency SSD - but is there any point to this technology?  There certainly isn’t any benefit putting them into shared storage architectures that we use today, where the additional performance benefit would be lost.  Why do we not have more SSD only arrays?

The above is only a subset of what happened over the last fortnight.  The storage industry never sleeps; things are always on the move and we’re never short of something to write about.  It’s good to see the array wars are not over and the competition between IBM, EMC, HDS, HP and others will continue to get tougher.  I think 2011 could be the year of features such as primary de-dupe and compression; clearly we are well positioned for some interesting times going forward.

In some respects this move makes sense for EMC, if Netapp are seen as their major competitor in this market.  However as we know, performing this kind of code merge isn’t simple and Netapp themselves have had problems achieving code delivery dates (posts here, here and here).  Perhaps these problems are typical of hardware vendors that are looking to squeeze more out of their increasingly legacy looking hardware.

Of course the biggest confirmation to this theory comes from moves by HP.  They have purchased many new niche storage vendors over the last couple of years, culminating with 3Par only last month.  This is a clear indication that their storage portfolio needed strengthening (and in places replacing) and they’ve acquired new technology to achieve this.

I doubt sales for EMC and Netapp are about to drop off a cliff any time soon, but I would suggest that continuing to lever new features into old architectures is a long term recipe for disaster.  Eventually EMC & Netapp will need a new approach.  Before everyone leaps on my comments, I’ll just remind you all that EMC promised discussed FASTv2 at EMC World in 2009. It has yet to be released as GA code on Symmetrix.  That’s 18 months people.  The world has moved on and FAST is not the only block-level tiering product in town, making it an also-ran rather than an innovation.  Maybe that’s the issue with legacy hardware.

Please Note:  Due to the high level of Spam comments I am receiving (even with captcha-style technology enabled), all commenters will be required to register.  Hopefully I won’t have to force this restriction for too long.

Update: EMC PR have contacted me and asked me to clarify some errors in this post

I have been asked to clarify by EMC PR that FAST wasn’t announced in April 2009 at EMC World but was actually announced in December 2009 as part of this press release.  This is not entirely true.  In fact, FAST was mentioned in the April 2009 release on V-MAX, which can be found here:

http://www.emc.com/about/news/press/2009/20090414-01.htm

The EMC bloggers also commented about it (as did I).  Barry Burke’s post is probably the most detailed:

http://thestorageanarchist.typepad.com/weblog/2009/04/1059-fully-automated-storage-tiering-fast.html

My post is here:

http://www.thestoragearchitect.com/2009/04/14/enterprise-computing-emc-announced-next-generation-v-max-architecture/

So, irrespective of the “official” release date, EMC were selling the FAST story from April 2009 onwards.  As for FASTv2, take a look at this article from Gestalt IT, dated May 2009:

http://gestaltit.com/featured/top/gestalt/emc-symmetrix-vmax-fast-virtual/

FASTv2 was being discussed at this point.  This wasn’t an “official” EMC announcement, but FASTv2 was being discussed at EMC World 2009.

Finally, I’ve been asked to clarify the fact that FASTv2 was GA as from August 24th. This is the press release I’ve been directed to:

http://www.emc.com/about/news/press/2010/20100824-01.htm

It isn’t very clear what the new features of FAST are from this post, and strangely, neither Barry Burke or Chuck Hollis chose to blog about it.  I believe the release only covered the mid-range products, as this Register article discusses:

http://www.theregister.co.uk/2010/08/24/emc_fast_2_flare_30/

So, agreed, FASTv2 is available for CLARiiON & Celerra, but not for Symmetrix.

I’ve tweaked the wording in the post to reflect the specific mid-range reference.  I’ll leave it to your own judgement as to whether you’d rather go with the “official” release dates or the “unofficial” release dates.

As you can imagine, with the release of a new product, there’s a lot to take in. Fortunately I’ve been privileged to be provided with information from both HDS and HP on the hardware prior to release.  Regardless of which vendor you take the hardware from, some of the highlights include:

• 2.5″ drive support
• Redesigned core processor architecture
• Overall 2x improved performance
• Reduced Power consumption
• Higher scalability
• Block level-tiering

If you want to get the low-down on the hardware itself, have a look at Nigel Poulton’s great post here.

What doesn’t seem immediately obvious is the difference in architecture between this and the predecessor USP V.  With everyone’s focus on speeds and feeds, this fundamental change can easily be overlooked.  I recommend you look at my first two posts on array architectures:

• Choosing Between Monolithic and Modular Architectures Part I
• Choosing Between Monolithic and Modular Architectures Part II

Part II will prove to be more useful for those people who are bound to want to compare the VSP/9500 with VMAX from EMC.  Rather than rush into a post today, I’ll be posting on the details over the next 48 hours, as there are software changes that also are significant in this release.  I’m also hoping to have some interviews in place, so look out for those too.

• Monolithic – this architecture is characterised by Hitachi USP, HP XP & EMC DMX and consists of a shared memory architecture and multiple redundant components.
• Multi-Node – these devices use loosely coupled storage “nodes” with a high-speed interconnect providing scalability by adding extra nodes to the storage “cluster”.  Products in this category include EMC VMAX and 3Par InServ.
• Closely Coupled Dual Controller – this is the typical “modular” storage architecture characterised by IBM DS8000, EMC CLARiiON, Hitachi AMS and HP EVA.
• Loosely Coupled Dual Controller - this category describes technology that are capable of device failover but aren’t closely coupled to enable individual LUN failover as the Closely Coupled model permits.  This category is characterised by arrays such as Netapp FAS filers and Compellent Storage Center.
• Single Controller – this category covers devices that act as standalone products, including SOHO storage devices such as the Iomega IX4 & Data Robotics Drobo series.

The above list isn’t exhaustive and it’s my own personal categorisation.  There are many more vendors of technology than I’ve listed here.  In addition, none of these lists qualify as “Enterprise” in their own right.  The use of this term is a hotly debated subject.

Monolithic Architectures

EMC DMX High Level Architecture

Monolithic arrays use a shared cache architecture to connect front-end storage ports to back-end disk.  This is shown clearly in the architecture diagrams shown here, representing the internal connectivity of the EMC DMX  and Hitachi USP storage arrays.  Each of the memory units is connected to each of the front-end directors and the back-end disk directors.  Hitachi divide their cache into two halves for Clusters 1 & 2 in the array; EMC have up to eight cache modules.  This architecture has positive and negative benefits; firstly having director connections connecting to all cache modules ensures resources aren’t fragmented;  unless cache becomes completely exhausted there’s always connectivity to another cache module to process a user request.  It also doesn’t matter on which port that request comes in; the cache module can process any request from any port to any back-end disk.  This connectivity is also beneficial in terms of failure.  If a cache module fails, for example, only the cache on that module is lost; in a fully deployed architecture the total cache would drop (by 1/8th in EMC’s case), but front and back-end connectivity would remain the same.  With this model it is possible pair up storage ports and have a single LUN presented from 1 or more ports with no performance impact; the path length between a storage port and disk adaptor will always be the same.

This any-to-any model also has disadvantages.  The connectivity is complex and therefore becomes expensive and requires overhead to manage and control the interaction between the various components.  In addition, there’s a limit to the practical scalability of this architecture.  With eight FE, BE and cache modules, there are 128 connections in place; (8x8x2).  Adding a single cache module requires an additional 16 connections; similarly, adding more front or back-end directors requires more connectivity.  Also monolithic arrays are based on custom components and custom design, increasing the ongoing maintenance and development costs for the hardware.

One other point to remember; front and back-end directors have their own processors.  It is possible for the traffic across the directors to be unbalanced and for some processors to be more heavily utilised than others.  I’ve seen a number of configurations where USP V FED ports are running at 100% processor utilisation due to to small block sizes.  This means manual load balancing is required both in initial host placement and subsequently as traffic load increases.  This fact is worth bearing in mind as we move to more highly virtualised environments as it is likely host port utilisation will start low and rise over time as more virtual machines are created.

Hitachi USP High Level Architecture

Now that the DMX platform has been put out to pasture in place of VMAX, it appears Hitachi are the only vendor continuing down the monolithic route.  Next time I’ll discuss Multi-Node arrays and why they may (or may not) be a replacement for today’s monolithic devices.