Background

Solidfire was founded in 2009 by Dave Wright, who previously had created Jungledisk, a cloud backup provider.  When Jungledisk was acquired by Rackspace, Dave had an epiphany around the way in which storage arrays were being used by cloud IaaS (Infrastructure as a Service) providers.  This is the genesis of SolidFire and forms the basis for some of the specific features the SolidFire arrays offer.

The Offering

SolidFire have taken a slightly different approach to similar vendors in this space and chosen to deliver their product as a cluster of computing nodes.  Each node contains processing, memory and disk in a fixed format, with scalability achieved by adding nodes in a clustered configuration.  Performance is claimed to grow linearly across the cluster, as data is spread across all cluster nodes for both capacity and I/O load.  This distribution mechanism provides for both consistent performance but also adds redundancy, with data replicated between nodes at the block level.  A failure in a disk or node is handled automatically, re-establishing data redundancy.

SolidFire currently sell one model of storage node, the SF3010.  This is a 1U “pizza box” rack-mount server with two Intel Xeon 2.4Ghz 6-core processors, 8GB of NVRAM (write cache) and 72GB of memory.  Each node has ten 300GB 2.5″ SSD drives for a total of 3TB of raw storage.  SolidFire have chosen the iSCSI protocol for front-end connectivity, with each node having two 10GbE connections.  This choice of protocol is probably due to the clustered design; Fibre Channel isn’t easy to cluster without suitable multi-path software on the host servers.  A single SolidFire cluster can scale from 3 to 100 nodes.  Clearly a single node is not enough to run a resilient system, hence the recommendation for a minimum of three as the smallest configuration.

With a large amount of processing power per node, data entering a SolidFire cluster is de-duplicated and compressed inline.  All LUNs are also thin provisioned.  None of these features are user configurable and are all enabled by default.  As a result, the 3TB raw per node translates to 12TB usable.  We are seeing other SSD array vendors making similar claims that the usable capacity is more than the raw capacity due to data optimisation techniques. Designing these features into an array from the outset (especially with lots of CPU performance, memory and fast disk access) is something traditional vendors will struggle to emulate.

Secret Sauce

Every vendor needs to have their own “secret sauce” and SolidFire are no exception.  It’s important to look at their target market, which is cloud-based service provision.  This can mean both internal and external clouds, however the key message is that traditional provisioning methodologies don’t scale and don’t fit in automated environments.  This is pretty easy to see this when looking at traditional storage management provisioning tools.  SRM software is focused on interaction with the administrator, providing only point in time current views of storage and very few tools do concurrent provisioning well.

SolidFire have taken the approach of developing a REST-ful API for array management.  This provides for all of the standard tasks of LUN creation, mapping and destruction with the ability to handle hundreds of API calls per second across a cluster.  API integration is essential for any organisation looking to develop an automated cloud-syle provisioning process and this is an area where traditional array vendors simply can’t compete.  API functionality can be integrated into existing processes and removes the need for large numbers of storage administrators – something that may be a worry to many, but we’re moving past the days of managing individual files and LUNs.

Another part of the “secret sauce” comes in managing I/O workload.  Traditional arrays work on the assumption that all I/O needs to be delivered as quickly as possible and in the order in which it is received.  There are some (but very few) arrays today that enable an administrator to prioritise workload.  At best, all that is achieved is literally that – prioritisation rather than quality of service (QOS).  I/O is still delivered as fast as possible, without regard for the service level needed for the I/O request.  SolidFire have addresses the QOS issue by allowing individual LUNs to have minimum, maximum and burst levels of performance applied.  This means LUNs created on day 1 of a new cluster deployment should receive exactly the same QOS and I/O performance when the cluster is fully loaded.

Deployment Scenarios

The ability to implement QOS features in cloud computing can’t be stressed highly enough.  The first wave of IaaS (infrastructure as a service) enabled functional deployment – that is, proving workload could be moved to the cloud.  The next wave will offer more QOS options other than simple processor and memory increments and storage will be one of those features.  SolidFire arrays enables cloud providers to deliver differentiated levels of performance without having to deploy multiple tiers of storage.  This is an important point.  As soon as storage tiering is implemented, then efficiency drops, as there are always tiers that remain partially used.    Block level tiering fixes some of these issues, but requires data to be moved around as part of performance re-balancing and still needs storage arrays to be monitored and assessed when adding additional storage.  In addition, traditional arrays deliver I/O as quickly as possible, which can result in servers receiving more throughput than expected when an array is lightly loaded, but lower performance over time, which can be perceived by the end user as a performance problem.

The SolidFire solution will definitely see deployment in those organisations who have adopted a service-based approach to delivering computing services.  With the API, QOS and node-scalable functionality, it is tailor-made for cloud deployments.

Summary

Solidfire started shipping in 2Q2011 with a $/GB price “similar” to that of traditional arrays.  As with other vendors, the use of inline compression and compaction is being used to achieve an aggressive price point.  Delivering for the cloud market is a smart move, as is the use of a scale-out node architecture that can grow as storage demand increases within an organisation.  Cloud deployments use templated configurations, so the ability to configure and map LUNs via an API with no user intervention fits with automated orchestration requirements.  I can see SolidFire arrays being widely used in many places over the next few years.

Additional Resources

Related articles:

• Who Will Be The First Solid State Vendor to Be Acquired?
• Solid State Arrays: Pure Storage Inc

The following video of Dave Wright was recorded at TFD#8.

Click here to view the embedded video.

Disclaimer:  I attended TFD#8 as an invited blogger.  My accommodation, some transportation and most meals were paid for.  I was not compensated for my time, nor required to blog on any of the presentations.  None of my blog entries, or other postings receive any pre-approval or viewings from vendors before publication.

Background

Pure Storage was founded in 2009 by John “Coz” Colgrove and John Haynes, who have their background in Veritas and Yahoo respectively.  More details on their backgrounds can be found on the bios page of Pure’s website.  Other members of the team have worked in companies such as Netapp, Decru and Sun, including Michael Cornwall, who was the lead flash designer at Apple for the iPod and iPhone.  So far, the company has raised around $55m from investors, the most significant of which has to be Samsung, who provide all of the flash drives used in Pure Storage’s products.  We’ll touch on the relative merits or disadvantages of this later.

The Offering

So what are Pure Storage offering?  Well, it’s pretty simple; an all-flash storage array at a $/GB price that’s cheaper than traditional storage.  Today that consists of two models, the FA-310 and the FA320.  The FA-310 is a single controller with one storage shelf, providing up to 140,000 4K random write IOPS.  The FA-320 doubles the storage capacity and increases write IOPS to 180,000.

Focusing on the FA-310, the controller is based on two 6-core Intel Xeon processors with 48GB of memory.  Back-end connectivity is 6Gbs SAS and 40Gbs Infiniband, while front end connections are only based on Fibre Channel at this time (4x 8Gbs SFPs).  Storage is provided by 22x 256GB MLC flash drives, giving a raw capacity of around 5.5TB.  It’s not surprising that Fibre Channel is the only protocol available on the first models.  FCoE doesn’t have the adoption rate and iSCSI wouldn’t suitable for the type of traffic this array can support.  However the controller is detailed as having one spare expansion port, so we can speculate whether that is planned to be for Ethernet in the future.

The connectivity between the controller and disk is less than that offered at the front end.  This may seem odd but it reflects on one of the key features of the Pure Storage arrays.  Data entering the system is compressed and de-duplicated before being stored on disk, improving the overall efficiency of the array and reducing the volume of write I/O to physical media. The ability to perform data reduction before storing on media is the main way in which an acceptable price point can be met.  This is something many other vendors are also doing as most customers are clearly fixated on the $/GB formula as the only way to measure acquisition cost.  Pure quote a ratio of anything from 5-20x reduction and as anyone familiar with data reduction technologies will know, your mileage will vary depending on the type of data consumed.

Secret Sauce

Ultimately though, there has to be something that make Pure Storage stand out from the competition.  During our Field Day visit, we were lucky enough to have a presentation from Coz, without slides, using just the whiteboard.  He detailed what is probably the most important piece of Pure’s technology, and that’s the way they manage the SSDs themselves.

Solid state drives are fickle devices.  Every write wears them out and much effort has been put into technologies (like wear levelling, write amplification) to extend their lifetime.  MLC devices now have a much better reliability than they did a few years ago, allowing them to displace SLC in enterprise technology.  Understanding how to manage SSDs is Pure Storage’s secret sauce.  They work closely with the SSD manufacturers to understand the best ways to read and write from the devices in order to gain both maximum performance and maximum lifetime.  At the presentation they even claimed never to have had an SSD failure, something that was met with surprise by the audience present!

Part of the SSD management involves the use of RAID-3D, technology which manages the RAID stripe distributions across the disks.  RAID stripes are varied dynamically based on workload and the drive responses.  This allows failing drives to be avoided, increasing their lifetime.  It also means I/O response times can be made more predictable, avoiding random I/O spikes seen with individual SSDs as features like garbage collection kick in.

It makes sense to understand the best way to manage SSDs and having a relationship with the vendor of those devices certainly helps.  My only concern is whether single supplier relationships are ever good, from a cost, reliability and supply perspective.  Only time will tell.

Deployment Scenarios

So where would you deploy this kind of high performance array?  I don’t think simply replacing your traditional storage with a Pure array is the right approach.  One of the benefits of shared storage is that I/O demand consists of peaks and troughs, periods of high and low demand from many servers.  This means it isn’t necessary to deliver 100% full I/O performance to all servers all of the time, but only to meet peak demand, which is considerably cheaper to achieve than meeting maximum demand.

This isn’t the Pure approach.  Their arrays are capable of delivering 2000 IOPS for every TB of storage, even at 10:1 compression.  It means that the server environment driving this storage needs to have high I/O requirements across every TB of data.  Otherwise, the array is never running at peak efficiency.  It could be said that if the $/GB cost is at a parity with traditional arrays, then should this matter?  I think it does matter because there’s a perception that flash is an expensive technology and irrespective of the effective $/GB cost after data reduction, many customers will still focus on the raw storage and the cost of the device.

Summary

Pure Storage have done a great job in delivering a technology that brings solid state performance at an acceptable $/GB price.  There are some key features (data reduction, SSD management) that make this technology really work.  We have seen presentations of high I/O workload that can easily be managed by the Pure storage arrays, while continuing to deliver sub 1-millisecond responses.  None of the big storage vendors have technology that delivers I/O bandwidth in a way companies such as Pure Storage can.  All-flash versions of traditional arrays don’t have the added intelligence to manage SSD failures and peformance spikes.  I can therefore see that very quickly one of the three letter storage companies will be looking to acquire Pure or one of their many competitors.  For now, they need to focus on finding the right niche for their product, while educating customers in metrics other than $/GB.

Additional Resources

Pure were one of the companies at TFD#8 that were well organised in providing various pieces of media.  I’ve included some of them here, including a link to the entire presentation from the day.  There’s also a business-card sized set of instructions that Pure claim as their user manual.  It’s a fun way of demonstrating how simple their technology can be.

• Pure Storage Presentation from TFD#8
• Pure Storage Flash Array Datasheet
•  Pure Storage Flash Array Manual – Front
•  Pure Storage Flash Array Manual – Back

Click here to view the embedded video.

Disclaimer:  I attended TFD#8 as an invited blogger.  My accommodation, some transportation and most meals were paid for.  I was not compensated for my time, nor required to blog on any of the presentations.  None of my blog entries, or other postings receive any pre-approval or viewings from vendors.

The device itself is very different.  First of all, the DroboPro accomodates up to 8 drives and so is orientated horizontally (otherwise it would fall over).  At the back, the cabling supports Ethernet, USB and Firewire.  Unfortunately the plugs are recessed and consequently removing the Ethernet cable is a pain as you can’t easily squeeze the retaining lug that releases the cable.  The power supply is now integrated into the unit and there’s no separate PSU brick.  This could present a problem if the power supply fails, as it isn’t obvious how (or even if) the integrated power supply could be replaced.  I expect that power is now integrated to make the DroboPro more datacentre friendly.  One last comment; the device is considerably more more noisy than its smaller brother, presumably because it has to cool twice as many hard drives.

• USB and iSCSI Configuration – it isn’t clear whether the Drobo Dashboard can configure the DroboPro over either the USB connection or the iSCSI connection or whether USB connectivity needs to be retained for future configuration tasks.  Be aware – unplugging the USB cable seems to cause the Drobo to immediately reboot.
• Reduced Management – if the DroboPro is to be connected to an ESX server, then the documentation indicates the USB cable should be removed.  What’s then not clear is how the  ongoing status of the device can be managed.  For example, at this point, where should the Drobo Dashboard be run from?  The lack of a permanent management console means downtime to check the status of the device, to see the current occupancy levels and the effect of adding in additional capacity.
• Lack of detailed Information – As yet, I haven’t managed to work out where or whether there are details provided on the utilisation of each configured LUN.  The DroboPro allows for up to 16x 16TB LUNs, which is a huge volume of data (although ESX only allows iSCSI LUNs up to 2TB).  I’d like to be able to work out which of those 16 volumes consumes the most physical compared to logical space, especially with ESX where the Drobo’s intelligent file system aware technology currently doesn’t apply.
• Multi-Drobo Management. I have more than one Drobo device now (DroboPro, DroboShare & Drobo).  The dashboard doesn’t appear to cater for multiple devices managed from the same machine.

Thin Provisioning Primer

The underlying concept of thin provisioning is pretty simple; provide storage resources to those requesting it only as they need it.

Think of a standard ‘thick provisioned’ environment.  As thick LUNs are created, the storage is assigned and mapped to that LUN to the full extent of the size requested.  See, the first graphic, which shows a RAID group of four 5GB drives.  I’ve assumed “RAID-0″ here for simplification, i.e. no RAID overhead.  Each LUN (coloured separately) is made up from a 1GB slice of the available disks.  Thick provisioning is great if the LUNs are all 100% allocated.  In that instance, 100% of the available physical space is used up.  However, it is never the case that 100% of a LUN is used and so wastage exists. 

Look at the second graphic.  This shows how thin provisioned LUNs work.  As storage is requested by the LUN, the space is mapped to physical blocks of storage.  In this example, none of the logical LUNs are fully utilised and so don’t consume their full theoretical capacity.  This means that the pool of space can be over-subscribed and a sixth new LUN created.  Obviously there’s no such thing as a free lunch or infinite storage resources and in this example if a further five blocks are requested then physical space would be exhausted.  The next request for a new storage block would result in an error situation and this represents the main concern with over-subscribing thin provisioned volumes.

Now we get the concept of thin provisioning, there are a further two aspects to consider.  Firstly, when we say a LUN isn’t 100% utilised, what to we mean?  Second, how can deleted blocks be returned to our free physical pool?

As LUNs are presented to hosts, they are formatted with a file system, for example on Windows it’s NTFS; a VMware environment would use VMFS.  The file system will have a standard layout which determines where the file index sits and the method in which files are allocated onto the disk.  Have a look at the third graphic.  This is a map of the C: drive for one of my servers.  Each block represents approximately 22MB.  You can clearly see the MFT (NTFS index) in the centre of the volume.  A large percentage of the disk is unused.  In a thin provisioned environment, storage would have been requested only for the blocks with valid data and in this way, a LUN can be less than 100% allocated. 

OK, so what happens if I create some files then delete them on the file system?  Most file systems remove a file by deleting the entry in the index rather than physically overwriting the file contents with binary zeros.  This is quick and efficient (if not slightly unsafe security wise).  The actual data isn’t overwritten and it is this ‘logical’ deletion that enables undelete utilities to work.  The trouble is, most disk arrays are not file system aware and so can’t detect the logical deletion of a file.  Those arrays that offer thin provisioning typically detect unwanted space by looking for blocks containing only ‘binary zeros’.  This means simply deleting files will not release unused space back to the free block pool (except for one storage device I’ll discuss in more detail another time, that’s Drobo). Arrays which are capable of recovering unused space need to see data overwritten in order to recover it.

This (finally) brings us to the cookie analogy.  Imagine cookies are my free pool blocks.  There are a number of ways in which storage arrays operate in handling thin provisioning – different cookie monster personalities:

• The Greedy Cookie Monster; grabs all the cookies he thinks he might eat, but never eats all of them and never returns any – this is the thick provisioning model.
• The Selfish Cookie Monster; only grabs cookies as he gets hungry but if he doesn’t eat them immediately, doesn’t give them back – this is thin provisioning with no zero block reclaim.  Eventually thin provisioning will become thick as all logical blocks in a LUN become mapped to physical storage.
• The Nice Cookie Monster; takes the cookies as he gets hungry but only returns uneaten cookies if asked – this is thin provisioning with manual zero block reclaim.  A manual process is required to zero out the unused space and to return it to the free pool.
• The Saintly Cookie Monster; takes the cookes as he gest hungry and offers them back immediately he realises he can’t eat them  – this is thin provisioning with automatic zero block or free space reclaim. 

So, of the storage arrays out there offering thin provisioning, which fit the various Cookie Monster personality types?  I’ll leave that for you to guess.

Next up was Nirvanix. Nirvanix are a “cloud storage” company, operating in the same space as Amazon S3. After providing an overview of the company, Peter Pistek provided details of CloudNAS 2.0 a software offering which enables a Linux or Windows server to act as a NAS device, storing the data directly in Nirvanix’s cloud. This is a product I’ve reviewed in the past and previously I’ve not been favourable towards it. Expect to see my review of the 2.0 version in the near future.

Day 2 lunch was provided by W Curtis Preston and was followed by “Mr Backup” taking us through his new venture – Truth In IT, a paid-for community portal to assist clients in evaluating storage products (initially backup).

The afternoon was taken up by a trip to Data Robotics, creators of the Drobo storage device. This session was probably my most anticipated of the week and I wasn’t let down. It was superb – for a number of reasons. Firstly, CEO Geoff Barrall took us through the product in more depth, answering some of the questions I’ve always had around the way Drobo’s BeyondRAID technology operates. This will become a post in its own right, expect to read it over the coming week.

The Field Day delegates then split into groups for a practical session, followed by a questionnaire, the winners of which would receive a Drobo each. The winners were; Team #1 – me, Rod, Devang and Simon! Now I have two Drobo devices for my data.

The most anticipated part of the Drobo session was the unveiling of two new Drobo products. Details are under embargo until November 23rd, however I can say the new devices will fill out the Drobo range very nicely. Expect a post on this as the embargo date is reached.

Day 2 wrapped up with dinner at a local italian restaurant and signalled the start of delegate’s long trips home.

In summary, the first Gestalt IT Tech Field Day has been a great success. The presenters have been open with their product information, keen to discuss details and helped to provide a lot of insight into their thinking and strategies. Fortunately, no-one fell into the “marketing slideware” trap. Expect to see posts over the coming weeks and days that delve into some of the details we’ve seen over the last two days. As a group we’ve covered a huge number of topics to even attempt to put into a single post.

Congratulations to Stephen Foskett for setting the event up. A great deal of thanks also need to go to Claire Chaplais and Sunshine Mugrabi, without whom the event wouldn’t have happened. One final thought; where and when will Tech Field Day 2 be?

http://www.flickr.com/photos/33818355@N07/sets/72157622693448519/

Disclaimer/Disclosure: The sponsors and presenters of the Gestalt IT Tech Field Day event have only paid for my accommodation and travel expenses in San Jose during this trip. I am not employed or paid by any of the sponsors/presenters to express my views. I have received a Drobo storage device as part of the competition held at Data Robotics during the Field Day.  I also received a nice Drobo body warmer & T-Shirt and a T-Shirt from 3Par, plus a number of assorted USB memory sticks.

From VMware, we moved on to 3Par and a presentation on the InServ hardware, with a follow up from Symantec. Again the parthership message was clear. 3Par provide thin provisioning with the ability to reclaim unused resources. This is achieved today through detecting patterns of data with no content (binary zeros), but in partnership with Symantec will be integrated into the Veritas File System. The 3Par presenters were excellent and the only disappointment was the lack of time to talk. To be fair, it would have been possible to take the whole day just discussing 3Par alone.

Next stop was the Computer History Museum for the first evening party. This was a chance to catch up more informally with the sponsors. It was also an opportunity for me to talk with The Notorious 3P (otherwise known as Marc Farley). The museum was fascinating, with exhibits ranging from Babbage’s Difference Engine to the first Apple I and a range of computing devices from my youth (the Commodore Pet, Sinclair Spectrum and many many others).

Festivities continued at the bar of the DoubleTree where the heated discussion turned to the subject of thin provisioning with VMware. it’s interesting that thin provisioning still causes serious debate.

All in all, the day was extremely successful, The depth and quality of the content in the presentations was very high; the whole day ran without a hitch, to which the organisers (Stephen, Claire and Sunshine) deserve all the credit.

In the coming weeks I’ll discuss in more detail the content of today’s presentations, including more talk about thin provisioning. In the meantime, here are pictures from the day.

http://www.flickr.com/photos/33818355@N07/sets/72157622685914563/show/

There are a select number of bloggers from the IT community attending (myself included), with interests covering Storage, Networks, Virtualisation and more. The reason the blogging community are interested I think is pretty obvious.  The question is, why would vendors want to do this?

It’s worth taking a step back to look at what the bloggers in question (who also write for GestaltIT) bring to the table.  No doubt there’s the insight and unbiased commentary but probably more important is the ability to access potential customers via a route which traditional marketing can’t deliver.  After all, the bloggers are independent, not paid by the vendors and have built their reputations and followings on this level of impartiality.  Clearly the vendors choosing to present their wares next month both believe in their products and believe that the blogging phenomenon is a better way to get the message out on their products in an impartial manner.

For me personally, I’ll be interested in looking at some technologies I’ve not really seen in detail (Ocarina, Xsigo) plus others I already know. Expect pictures (perhaps some video) and lots of commentary over the course of the two days (November 12/13).