Lotus Organiser

Lotus Organiser

As we will see, many personal organisers have had their origins or been developed by British companies.  Lotus Organiser was developed by UK company Threadz and acquired by Lotus, being sold as part of their SmartSuite package.  The software mimicked the look of a paper organiser, with images for calendar, contacts and so on.  Although I liked Lotus Organiser, I remember it as being buggy and being annoyed by the inability to easily move data between formats and platforms.

Psion Organisers

Psion, another British company, released their handheld organiser in the mid 1980′s.  Looking like a rugged calculator, the Psion Organiser II model I owned, came with a two line screen and plug-in memory modules or RAMPAKs.  Style wasn’t one of the Psion Organiser’s strong points, with an A-Z keyboard and solid shell cover.  However it was an electronic device that was to a certain extent, portable.

Psion Series 5

Psion followed the initial Organiser models with the Series 3 and 5.  I had both, but my favourite was the Series 5; I owned both the 5 and 5mx versions.  The Series 3 and 5 models were elegant with their clamshell open and close design, ran for a month on just two AA batteries and could be read outdoors in bright sunlight.  Ultimately however, they had two fatal flaws; first there was never a colour model (the drain on battery life I assume deemed too much of a compromise) and synchronisation with Windows was flakey at best and useless at its worst.

Windows CE Devices

iPAQ hx4700

What Psion didn’t do, devices like HP’s Jornada, based on Microsoft Windows CE did.  They offered colour screens and better PC integration, but suffered with performance, screen quality and battery life.  I owned a number of Windows CE-based devices, including a Jornada 540 (with a massive 16MB of memory), an iPAQ hx4700 and the tediously dull iPAQ hx2790.  The hx4700 was great as it had a large screen, but I dropped it onto a concrete floor and to my chagrin, the device didn’t bounce and wasn’t replaceable as HP had stopped making them.

The Rise of Apple

2007 was probably my low point of PDA ownership.  I’d tried other alternatives like the Palm Pilot, but then I happened on the iPod Touch.  This evolved into owning three iPhones and today the iPhone I have does everything I need.  The modern version of the organiser (whether Android or iOS-based) goes right back in functionality to the original paper organiser I first had, except apps have replaced the different types of removable pages a Filofax offers.  In essence both “devices” are not dissimilar, both being extensible from their initial configuration.

Best of the Rest

Some of the ones that got away, which I’d liked to have purchased but couldn’t afford included the enigma that was the Apple Newton, which looking back now was a genius of a device, that was too far ahead of it’s time.  There was also the Nokia 9000 Communicator.  That almost brought together the best of Psion and a mobile phone but again, was outside of my price bracket at the time.

The Future

What’s the future going to be in the next 25 years?  I’m sure the Filofax will still exist in paper form.  However electronics is both fast moving and totally unpredictable past 2-3 years ahead.  It’s easy to suggest faster connectivity and better quality displays, but past that, perhaps the next wave of improvements will be software-based, combining information, intelligent and predictive search and data presentation to the user.  Personally I can’t wait!

Hitachi Non-Disruptive Migration

It’s probably fair to say that data migration is one of the most tedious tasks in storage management.  Moving data from one array to another, simply to decommission a piece of hardware or for load and capacity balancing, doesn’t ever inspire me. But it’s a necessary evil and one I’ve done many times.  Unless the existing storage configuration has been well planned, migrations can take months to accomplish, at high cost, in order to avoid outages and maintain data integrity.

So during last year’s Hitachi Influencers event in San Jose, I was extremely interested to see that the migration issue might have finally been put to bed by one of the vendors.  This week we saw the release of last year’s demo; Hitachi’s non-disruptive migration service.

At it’s heart the concept isn’t that complicated; by enabling a new target storage array to act as if it is another set of data paths through the SAN to the original array, a LUN can be mirrored away from the source, onto the target, while the data is mirrored in the background.  Once the mirroring is complete, drop the first set of paths that connect to the LUN and in effect the LUN “migrates” to the new target array in a completely transparent fashion.

This migration process works because Hitachi are able to both virtualise a LUN through another storage array (technology that has existed for quite some time) and move the World Wide Name (WWN) of the source array across and present it out from the target.  In essence, the target array “spoofs” the host into believing that the WWN of the old array still exists on the network.

What’s interesting here is that the migration process is making use of standard Fibre Channel protocols and so the migration process could be used to move data from any array, not just Hitachi products.  There’s also no disruption to the existing configuration, as the virtualisation of the source array can be achieved behind the scenes without impacting the existing configuration.  There are of course some restrictions or issues to consider; migrating LUNs that are array replicated needs to be thought through and of course the target array continues to appear with a different WWN, which could be confusing during and after migration work.

However, the power of this technology is the ability to avoid cost.  Hitachi’s storage economist, Dave Merrill, estimates that storage costs around $7K-15K per TB to migrate between arrays, which is way more than new storage costs to acquire in the first place.  Consider that many customers will put a new vendor on the hook to cover migration, then this service puts Hitachi in a much stronger position when tendering for swap-out business.

As with everything, what’s usually more interesting is not what can be achieved today, but what can be done tomorrow.  The ability for an Hitachi array to offer out virtual WWN port names means that an entire physical array could be split into multiple virtual arrays, in a similar way to the Multistore feature offered by Netapp.  Now, instead of having WWNs that match to a physical device, a virtual array could be created.  This virtual array can be managed with its own QOS, or migrated or shared between hardware platforms, without the user having to have any knowledge of where the data is actually sitting.  Imagine using a single physical array to create secure multi-tenant virtual arrays with the ability to manage each with an individual QOS (something that Multistore can’t do today).

Although I can’t claim to have a crystal ball, I did predict this kind of feature on the release of the VSP in 2010 (see my previous article where I discuss this possibility).  I have no idea whether Hitachi will deliver this feature, but I hope they do.

My only disappointment with Non-Disruptive Migrations is that it may have come to late for many organisations.  As virtualisation becomes more prevalent, migrations will be achieved using (for example) Storage vMotion in the hypervisor, negating the need to care whether the array can perform the migration on the host’s behalf.  However server virtualisation isn’t everywhere and the option of multi-tenancy is still a powerful one, even with the ability to virtualise the server.

Related Posts

• It’s About Time (and Money)
• No More Migration Tears

Disclaimer: Last year I attended Hitachi’s Influencer Forum in San Jose.  Hitachi paid for my travel and accommodation as well as most meals.  Most of the content of this event was NDA only and so hasn’t been discussed until now.  There is no requirement on me to blog about any of the content presented during the event.  I am not otherwise employed by Hitachi, or compensated for my time.

Lightning AKA VFCache

VFCache (Very Fast Cache) is the final product name for the project that was called Lightning.  It turns out that this offering is nothing more than a PCIe SSD card for servers (not all servers mind you, but at this stage quite a few).  The initial offering combines with software to act as a very fast read cache to the host.  In Windows terms, this is implemented as a filter driver that sits above the STORPORT driver in the I/O stack, with similar implementations on Open Systems platforms.  The software component of VFCache tracks I/O and caches reads in order to speed up future I/O requests without needing to go to external disk.  Writes to disk are not cached by VFCache and EMC tried to make a virtue of the fact their product acts as a “write-through” cache, meaning I/O writes have to be committed to physical disk before the cache acknowledges them to the host.  Rather than being a benefit, write through mode in this instance is more likely to make the cache less effective by polluting the cache with writes that can’t be released until confirmed externally.  When there’s a difference in I/O if microseconds versus milliseconds, then this difference really matters.  However, I don’t think this is a design flaw, merely a placeholder for the future, as I’ll discuss later.

Disappointingly for EMC, VFCache 1.0 really is a 1.0 version in terms of feature support.  Within VMware ESXi for instance, the card installs with a device driver that only allows the cache benefits to be used when the filter driver is deployed into each ESXi guest, so it’s not simply a case of insert the card and off you go.  Moreover, the VFCache appears as a DAS device within VMware and so can’t be used in conjunction with vMotion.  For many organisations this is a huge omission as there’s a big correlation between high performance and high availability; the lack of vMotion isn’t acceptable.

The Competition

We can’t go any further on the VFCache discussion without mentioning the competition and in one of the presented slides, EMC paid homage to the market leaders, Fusion-IO.  Their ioCache product already accelerates VMware ESXi and Windows 2008 environments, using a similar hypervisor plugin approach.  ioCache already offers double the capacity of VFCache and it’s likely Fusion-IO have larger capacity cards in the pipeline as they already offer a range of SLC and MLC flash devices.

Thunder Follows Lightning

Surely Hitachi must be enjoying the irony of EMC choosing product code names based on already defunct HDS hardware (Thunder and Lightning were the mid-range and Enterprise products respectively that preceeded AMS and USP).  The next product announcement moves the flash-in-server story forward and explains how this technology is limited in terms of availability.  The move to centralised SAN environments was done precisely to fix the issues that occur with server-side SSD today.  Data is locked into the server, is difficult to expand (requiring downtime and physical intervention) and is isolated from access if a physical failure of the server should occur.  So, step up Project Thunder, EMC’s purpose build all-flash array.  This device allows multiple servers to share data across what EMC are calling the “server-area network”.  What they mean is a physically local, high-speed interconnect (such as Infiniband or Rapid-IO) between the server and a fast all-flash storage array.

The interconnect technology between devices already exists today (as already mentioned) but placing it into the server and using this for shared storage presents more of a challenge.  Where Lightning was a simple Filter Drive, Thunder will require deeper integration in order to manage consistency across all connected servers.  This isn’t something that comes new to EMC – think of how VMAX nodes interconnect and you have the model already there.  However, the implementation may require closer co-operation with server vendors than EMC can achieve, those same companies they are already in competition with for storage – IBM, HP and Dell.  This could mean Thunder becomes a VCE only product or is severely restricted when deployed in other manufacturers hardware.  We will have to wait and see. (Side Note: This also means that other all-SSD array manufacturers could become more attractive to HP, IBM & Dell as acquisition targets – check out here and here).

The Competition

Let’s not forget that dedicated all-flash arrays are already out there.  Recently I’ve discussed Pure Storage and SolidFire and there’s also Violin Memory, who have been going at this market for quite some time.  They already have the SSD array technology to a mature level including support for Infiniband; all that’s needed is a software driver to bring clustering to their products.

Futures

What can we expect going forward?  There are lots of gaps the product releases we’ve seen today.  Lack of vMotion support and no write-back cache to name only two.  The question we should be asking is what could be delivered in the future.  EMC have access to every piece of the I/O stack, from the hypervisor, the multi-path driver, through to the array.  Using PowerPath, EMC can develop more intelligent algorithms that choose whether to cache I/O locally in the server/hypervisor, destage to the array, leverage pre-fetching from disk and other clever ways to squeeze the best level of performance out of the hardware stack.

Conclusions

EMC have ratified both the PCIe SSD and dedicated flash array markets with their announcements of Lightning and Thunder.  At this stage they are bringing only “me too” products to the market, with other vendors out there having already delivered more advanced technology than being announced today.  However EMC have two big advantages; (a) they are a huge organisation, with access to the majority of customers in the market and a great marketing team.  They have the ability to place their products into customer environments and use price as the main differentiator (b) they have a huge R&D budget and never stand still on product development.  Today’s 1.0 releases will be superseded within months and address some of the shortcomings we can see today.  The future battle will not be over the hardware, but the software that integrates I/O in the server to I/O on the array, delivering the benefits of local flash with the safety of external storage.  The eventual winner will be the vendor who gets that software and hardware integration right.

Related Articles

You may be interested in the following related articles from this and other sites.

• Fusion-IO Shares Tumble as New Entrants Prepare to Enter The Market
• Emulex – Evolution of the HBA
• Solid State Arrays: SolidFire
• Solid State Arrays: Pure Storage Inc
• Who Will Be The First Solid State Array Vendor To Be Acquired?
• HP & Violin?
• Enterprise Computing: Violin Memory Inc Release New All-SSD Array
• EMC VFCache (aka “Project Lightning”) Is One Small Step, But an Important One (Stephen Foskett)
• Cache Splash (Storagebod)
• VFCache Means Very Fast Cache Indeed (Chuck Hollis)
• VFCache: Hello World! (…and covers come off Project Thunder) (Chad Sakac)
• My take on EMC’s project lightning (Enrico Signoretti)

Enjoy!

This isn’t the first reality check of its kind; think about the untouchable STEC in the SSD market who’s shares once traded at almost $45 and are now just under $10.  Their demise was competition and the reliance of a single customer, EMC, who decided one day they had enough supply, thanks, as the SSD market wasn’t evolving as quickly as they had planned.  It seems the spectre of EMC is about to strike again, as Project Lightning, EMC’s PCIe SSD device draws close to release on 6th February this year.

Is it all bad for Fusion-IO?  Probably not, this is nothing more than a correction of an overpriced stock and doesn’t mean the company is in trouble.  Fusion-IO continues to innovate as we heard recently with the announcement of their Auto Commit Memory feature, which extends memory onto the PCIe bus using kernel-based code changes.  The idea is an interesting one; extend what in the old days would have been termed “real memory” using the next fastest level of storage, PCIe SSD.

Technically, I don’t think this change is that difficult to achieve.  Modern operating systems already have interrupt-based memory management, using virtual memory pages that are swapped into real memory as required.  Although I’ve not seen the details, Fusion-IO could have either used their PCI-SSD device as very fast cache paging device, rather than disk, or extended the available physical page slots the host can see.  It seems that the former rather than the latter is more likely, as the literature released discusses “flushing in-flight data”, something that’s not relevant with volatile physical memory slots.

Back to the competition; EMC may be coming into this market, however their product will be a 1.0 release.  Nevertheless, EMC already own the storage array and hypervisor, so it’s not a difficult leap to imagine that the first target for their Lightning technology will be in accelerating VMware ESXi deployments.  This means they would already be enhancing the kernel of that platform, giving them a great headstart on others in the marketplace.  As we see move towards the majority of server deployments being virtualised, the hypervisor is the place for this technology to sit.  Will VMware be willing to provide FusionIO and others the same kernel-level access as EMC must surely be getting?  Who is working with Microsoft to deliver this technology into Hyper-V?

In many respects, PCIe SSD devices are only at the very early stages of development.  In the coming months and years we are going to see some very interesting times indeed.

However, the VNX is still a CLARiiON at heart (with all the legacy baggage that entails), so one wonders what additional capabilites the VNX5500-F has to cope with this huge I/O workload and of course, to manage the finite lifetime of SSD devices.  More important is the ability to cope with the fickle performance of I/O spikes that are associated with SSD garbage collection.  There’s no mention of how (or even if) EMC have added technology to cater for these issues.  Bearing in mind what an all-flash array will cost, then 100% guaranteed low latency of every I/O will be expected.

The new market startups (Violin, Pure, SolidFire & others) will have to compete against EMC’s marketing machine but to be fair this is a technology that already offers a wide range of features, including connectivity via all the common protocols in use today.  EMC will be able to sell simply on feature, functionality and support.

The VNX5500-F may seem like putting a Rolls Royce engine in a mini compared to the competition, who are more like thoroughbred Ferraris and Lamborghinis, however as usual, cost will be the ultimate decider.  EMC don’t quote price, but simply indicate that cost per TPM is vastly reduced.  I’d like to see some real world list costs from EMC (which won’t happen) plus some statements on how this dedicated VNX deals with the particular issues of SSD drives.  If you are considering an all flash array, then these questions need to be on your list.

Background

Solidfire was founded in 2009 by Dave Wright, who previously had created Jungledisk, a cloud backup provider.  When Jungledisk was acquired by Rackspace, Dave had an epiphany around the way in which storage arrays were being used by cloud IaaS (Infrastructure as a Service) providers.  This is the genesis of SolidFire and forms the basis for some of the specific features the SolidFire arrays offer.

The Offering

SolidFire have taken a slightly different approach to similar vendors in this space and chosen to deliver their product as a cluster of computing nodes.  Each node contains processing, memory and disk in a fixed format, with scalability achieved by adding nodes in a clustered configuration.  Performance is claimed to grow linearly across the cluster, as data is spread across all cluster nodes for both capacity and I/O load.  This distribution mechanism provides for both consistent performance but also adds redundancy, with data replicated between nodes at the block level.  A failure in a disk or node is handled automatically, re-establishing data redundancy.

SolidFire currently sell one model of storage node, the SF3010.  This is a 1U “pizza box” rack-mount server with two Intel Xeon 2.4Ghz 6-core processors, 8GB of NVRAM (write cache) and 72GB of memory.  Each node has ten 300GB 2.5″ SSD drives for a total of 3TB of raw storage.  SolidFire have chosen the iSCSI protocol for front-end connectivity, with each node having two 10GbE connections.  This choice of protocol is probably due to the clustered design; Fibre Channel isn’t easy to cluster without suitable multi-path software on the host servers.  A single SolidFire cluster can scale from 3 to 100 nodes.  Clearly a single node is not enough to run a resilient system, hence the recommendation for a minimum of three as the smallest configuration.

With a large amount of processing power per node, data entering a SolidFire cluster is de-duplicated and compressed inline.  All LUNs are also thin provisioned.  None of these features are user configurable and are all enabled by default.  As a result, the 3TB raw per node translates to 12TB usable.  We are seeing other SSD array vendors making similar claims that the usable capacity is more than the raw capacity due to data optimisation techniques. Designing these features into an array from the outset (especially with lots of CPU performance, memory and fast disk access) is something traditional vendors will struggle to emulate.

Secret Sauce

Every vendor needs to have their own “secret sauce” and SolidFire are no exception.  It’s important to look at their target market, which is cloud-based service provision.  This can mean both internal and external clouds, however the key message is that traditional provisioning methodologies don’t scale and don’t fit in automated environments.  This is pretty easy to see this when looking at traditional storage management provisioning tools.  SRM software is focused on interaction with the administrator, providing only point in time current views of storage and very few tools do concurrent provisioning well.

SolidFire have taken the approach of developing a REST-ful API for array management.  This provides for all of the standard tasks of LUN creation, mapping and destruction with the ability to handle hundreds of API calls per second across a cluster.  API integration is essential for any organisation looking to develop an automated cloud-syle provisioning process and this is an area where traditional array vendors simply can’t compete.  API functionality can be integrated into existing processes and removes the need for large numbers of storage administrators – something that may be a worry to many, but we’re moving past the days of managing individual files and LUNs.

Another part of the “secret sauce” comes in managing I/O workload.  Traditional arrays work on the assumption that all I/O needs to be delivered as quickly as possible and in the order in which it is received.  There are some (but very few) arrays today that enable an administrator to prioritise workload.  At best, all that is achieved is literally that – prioritisation rather than quality of service (QOS).  I/O is still delivered as fast as possible, without regard for the service level needed for the I/O request.  SolidFire have addresses the QOS issue by allowing individual LUNs to have minimum, maximum and burst levels of performance applied.  This means LUNs created on day 1 of a new cluster deployment should receive exactly the same QOS and I/O performance when the cluster is fully loaded.

Deployment Scenarios

The ability to implement QOS features in cloud computing can’t be stressed highly enough.  The first wave of IaaS (infrastructure as a service) enabled functional deployment – that is, proving workload could be moved to the cloud.  The next wave will offer more QOS options other than simple processor and memory increments and storage will be one of those features.  SolidFire arrays enables cloud providers to deliver differentiated levels of performance without having to deploy multiple tiers of storage.  This is an important point.  As soon as storage tiering is implemented, then efficiency drops, as there are always tiers that remain partially used.    Block level tiering fixes some of these issues, but requires data to be moved around as part of performance re-balancing and still needs storage arrays to be monitored and assessed when adding additional storage.  In addition, traditional arrays deliver I/O as quickly as possible, which can result in servers receiving more throughput than expected when an array is lightly loaded, but lower performance over time, which can be perceived by the end user as a performance problem.

The SolidFire solution will definitely see deployment in those organisations who have adopted a service-based approach to delivering computing services.  With the API, QOS and node-scalable functionality, it is tailor-made for cloud deployments.

Summary

Solidfire started shipping in 2Q2011 with a $/GB price “similar” to that of traditional arrays.  As with other vendors, the use of inline compression and compaction is being used to achieve an aggressive price point.  Delivering for the cloud market is a smart move, as is the use of a scale-out node architecture that can grow as storage demand increases within an organisation.  Cloud deployments use templated configurations, so the ability to configure and map LUNs via an API with no user intervention fits with automated orchestration requirements.  I can see SolidFire arrays being widely used in many places over the next few years.

Additional Resources

Related articles:

• Who Will Be The First Solid State Vendor to Be Acquired?
• Solid State Arrays: Pure Storage Inc

The following video of Dave Wright was recorded at TFD#8.

Click here to view the embedded video.

Disclaimer:  I attended TFD#8 as an invited blogger.  My accommodation, some transportation and most meals were paid for.  I was not compensated for my time, nor required to blog on any of the presentations.  None of my blog entries, or other postings receive any pre-approval or viewings from vendors before publication.