The Storage Architect

Nekkid Tech at Dell Storage Forum London

admin — Thu, 02 Feb 2012 21:41:50 +0000

Last month I briefly took part in Nekkid Tech Podcast #14. This was recorded in London at Ye Olde Cheshire Cheese Pub on Fleet Street. The evening itself was quite entertaining and loud as you can hear from the podcast. Greg and Ed interviewed a number of the Dell management team on what [...]

Fusion-IO Shares Tumble as New Entrants Prepare to Enter The Market

admin — Fri, 27 Jan 2012 07:33:12 +0000

It’s never a good day at the office when your shares drop by 20% in a single day. However the relentless drive to produce better results quarter-on-quarter is what the US stock markets demand of publicly listed companies such as Fusion-IO. Poor results this week saw their shares punished by a drop from just over $30 to below $25 today.

This isn’t the first reality check of its kind; think about the untouchable STEC in the SSD market who’s shares once traded at almost $45 and are now just under $10. Their demise was competition and the reliance of a single customer, EMC, who decided one day they had enough supply, thanks, as the SSD market wasn’t evolving as quickly as they had planned. It seems the spectre of EMC is about to strike again, as Project Lightning, EMC’s PCIe SSD device draws close to release on 6th February this year.

Is it all bad for Fusion-IO? Probably not, this is nothing more than a correction of an overpriced stock and doesn’t mean the company is in trouble. Fusion-IO continues to innovate as we heard recently with the announcement of their Auto Commit Memory feature, which extends memory onto the PCIe bus using kernel-based code changes. The idea is an interesting one; extend what in the old days would have been termed “real memory” using the next fastest level of storage, PCIe SSD.

Technically, I don’t think this change is that difficult to achieve. Modern operating systems already have interrupt-based memory management, using virtual memory pages that are swapped into real memory as required. Although I’ve not seen the details, Fusion-IO could have either used their PCI-SSD device as very fast cache paging device, rather than disk, or extended the available physical page slots the host can see. It seems that the former rather than the latter is more likely, as the literature released discusses “flushing in-flight data”, something that’s not relevant with volatile physical memory slots.

Back to the competition; EMC may be coming into this market, however their product will be a 1.0 release. Nevertheless, EMC already own the storage array and hypervisor, so it’s not a difficult leap to imagine that the first target for their Lightning technology will be in accelerating VMware ESXi deployments. This means they would already be enhancing the kernel of that platform, giving them a great headstart on others in the marketplace. As we see move towards the majority of server deployments being virtualised, the hypervisor is the place for this technology to sit. Will VMware be willing to provide FusionIO and others the same kernel-level access as EMC must surely be getting? Who is working with Microsoft to deliver this technology into Hyper-V?

In many respects, PCIe SSD devices are only at the very early stages of development. In the coming months and years we are going to see some very interesting times indeed.

What to Watch in Storage for 2012

admin — Tue, 24 Jan 2012 08:57:25 +0000

As I write this post, 2012 is already 6.3% completed. So, for the remaining 93.7% of the year, what can we expect the storage themes to be? What will companies be focused on and what can we expect from the vendors?

Data Reduction

The recent flooding in Thailand has had everyone on high alert about the rise in disk prices, see this Register article as a good example. We are normally used to seeing disk prices decline over time, however the start of this year may see exactly the opposite. Suddenly all those data optimisation (aka clean up) tasks you didn’t think were worth it, now come back into view as they may start to become cost effective. Although the “do more with less” mantra keeps getting pushed year after year, in reality, cleaning up and keeping a storage environment tidy does have to be considered in terms of return on effort. There’s not a lot of financial gain to be had from spending time recovering unused resources when you are only 50% utilised. With configurations, perhaps it’s easy to be on top of everything; not so when you start getting into multi-petabytes. Expect to see more push on the optimisation techniques – data de-duplication being the most likely new contender for the storage admin’s hard-pressed time. It’s worth remembering though that thin provisioning still hasn’t gained the reach it was expected to, or should have, if vendor surveys are to be believed. Maybe a focus on the basics is a good thing.

Big Data

I don’t like it when marketing departments start coining IT terminology and “Big Data” is one of them. I expect to see more companies pushing the benefits of using all those “unloved terabytes of information you didn’t think you needed” by turning them into “intellectual property” ripe for exploitation, but in reality I think the benefits are overplayed. If there’s an opportunity to use data in clever ways, then organisations will do it, irrespective of the technology available. There’s also not a lot of commercial sense in putting all your data into an expensive Data Warehouse on the expectation that someone will think of something useful to do with it sometime in the future. Perhaps that’s what Big Data should be renamed; Something, Someone, Sometime.

Niche Technologies

I think we will see more niche technologies in the coming year. We have Violin, Pure Storage, SolidFire, Nutanix, Virsto and many others producing technology that fit specific requirements, whether it be high performance, utility or hybrid virtualisation & storage products. Expect to see more of these and potentially the first acquisition by one of the big storage vendors. There’s also an outside chance we’ll see a bigger take up in virtual storage arrays. VMware’s VSA has brought credibility to that marketplace and there are already many established players like Nexenta.

Push to The Cloud

No summary would be complete without a discussion on Cloud. We have pure plays in the cloud space but there are many hybrid offerings, such as Nasuni (currently under review in the TSA Lab), StorSimple, CTERA and others. These companies are interesting as they start to make cloud storage a viable option by reducing the latency with local caching. I think these kinds of products will act as a gateway for many organisations to reduce their footprint and dependence on managing existing “legacy” filer technologies.

I’d be interested to hear everyone’s opinion on their views for 2012, so feel free to comment, good or bad, but always courteous.

Drive Prices Increase – Who Will Suffer Most?

admin — Sun, 06 Nov 2011 21:24:21 +0000

Storagezilla calls out hard disk availability in his recent post. In fact, I discussed the subject of drive prices last week with James Carter of Touchdown PR on my way to SNW Europe. James highlighted he was seeing prices on standard drives having doubled recently.

I went back and looked at my last drive purchases – two Western Digital 2TB SATA-II drives at around £65 each. Currently these are now retailing on Amazon for £150 each, with the latest 2TB drives around the £100 mark.

Prices have clearly risen sharply due to the flooding in Thailand, however I wonder, which vendor suffers the most in this scenario?

The most obvious answer would appear to be that the vendors with the least margin will suffer most. Think of the likes of Drobo or Overland, who sell relatively low cost hardware. Theoretically vendors such as EMC and Netapp should be able to swallow the additional cost, especially if they are already charging 10-15 times the underlying raw cost of the drive on a per terabyte basis. The drive cost makes up a small part of their overall price. The big vendors should also have the muscle to fulfill their demand first with what supply is available.

However I don’t think things are that simple. Drobo arrays can be purchased without disk drives, allowing them to use whatever drives are available, so drive prices won’t hurt bare array sales. These arrays also allow mix and match and to upgrade per drive, so are potentially more flexible.

Bear in mind savvy customers of EMC and others will have negotiated quarterly price erosion, so those on existing deals will probably not see much change. I would expect the big boys to have hedged their supplies through multiple supply lines and stockpiles but their margins will be affected if supply problems persist.

It’s also worth thinking about efficiency at this point. If you’re not using thin provisioning or other data reduction technologies, then you should be. If your vendor doesn’t offer it, then there are plenty out there who do. As prices rise, it may be time to look again at implementing these features and fixing the processes that stop you using them today.

One last thought. Are NAND prices being affected? If not then the SSD array manufacturers must be rubbing their hands together with glee.

Emulex – Evolution of the HBA

admin — Fri, 04 Nov 2011 10:11:12 +0000

At SNW Europe this week I took some time to talk to Emulex about their OneCommand product. It’s been upgraded to version 2.0, which will be the subject of a post for another day, once I’ve had a chance to install and review it.

I also discussed the core of Emulex’s business, namely HBAs (Host Bus Adaptors) and it’s time for a little speculation.

We’re all familiar with the way HBAs work; they bridge the internal bus to external devices, the most obvious example being Fibre Channel. Most recently companies such as Fusion-IO have moved storage closer to the CPU by putting flash storage directly onto the PCIe bus. However there are issues with using PCIe flash cards, most obviously the fact that the data is tied to the server in which the card is stored. If the server fails then data recovery becomes a protracted issue involving physical intervention within the hardware. So, although PCIe flash cards are good for handling write data, they present a consistency problem in ensuring writes are stored elsewhere in case of hardware failure.

With a long history in developing HBAs, Emulex are well placed to exploit this weakness and create a new type of device, a hybrid between the HBA and the flash card.

Imagine a device that looks and functions like a traditional HBA, but also has flash storage. It would be possible to accelerate I/O destined for spinning disks by using the flash as both a read and write cache. Now I/O performance improves to the benefit of flash speeds, yet can be destaged to external storage for additional security and replication elsewhere. All this gets handled by the HBA, independently of the host and with processing offload.

There would be obvious issues with the synchronicity of data writes, especially with shared LUNs (although that could be overcome with clever use of SCSI reserves) and multipathing. However, there are other possibilities; random I/O could be staged on flash and written to physical disk to reduce the randomness and improve throughput. I/O packets could be analysed for data traffic management (to improve the way OneCommand works). The HBA could also be used as a physical data splitter, to enable CDP without the delay penalty of implementation in the O/S or with latency.

All these ideas are pure speculation and Emulex may or may not be working on this technology. My bet is that they are working on something, as probably are Qlogic and Brocade.

The most interesting thing is what we’ll see with PCIe flash cards. This market only had one main contender – Fusion-IO. However as we saw with STEC, who owned the enterprise SSD market, things change. There will very quickly be many competitors to Fusion-IO and there will need to be some advancement of the technology to keep ahead of others in the market. Hybrid cards may offer just that leap forward.

EMC Releases All Flash VNX

admin — Mon, 31 Oct 2011 09:51:56 +0000

As the vendor who introduced flash drives into traditional arrays some 2+ years ago, we shouldn’t be surprised that EMC has released an all-flash version of its mid-range array, the VNX. The new device, codenamed VNX5500-F can support up to 250 200GB drives for a maximum all-flash capacity of 50TB. EMC don’t say, but assuming the drives are something like the STEC ZeusIOPS SSD, with around 25,000 8K block IOPS on a 70/30 read/write split, then we’re looking at 625,000 IOPS per shelf and around 4.8GB/s throughput (again, per shelf).

However, the VNX is still a CLARiiON at heart (with all the legacy baggage that entails), so one wonders what additional capabilites the VNX5500-F has to cope with this huge I/O workload and of course, to manage the finite lifetime of SSD devices. More important is the ability to cope with the fickle performance of I/O spikes that are associated with SSD garbage collection. There’s no mention of how (or even if) EMC have added technology to cater for these issues. Bearing in mind what an all-flash array will cost, then 100% guaranteed low latency of every I/O will be expected.

The new market startups (Violin, Pure, SolidFire & others) will have to compete against EMC’s marketing machine but to be fair this is a technology that already offers a wide range of features, including connectivity via all the common protocols in use today. EMC will be able to sell simply on feature, functionality and support.

The VNX5500-F may seem like putting a Rolls Royce engine in a mini compared to the competition, who are more like thoroughbred Ferraris and Lamborghinis, however as usual, cost will be the ultimate decider. EMC don’t quote price, but simply indicate that cost per TPM is vastly reduced. I’d like to see some real world list costs from EMC (which won’t happen) plus some statements on how this dedicated VNX deals with the particular issues of SSD drives. If you are considering an all flash array, then these questions need to be on your list.

Solid State Arrays: SolidFire

admin — Wed, 26 Oct 2011 10:01:53 +0000

One of the attendees at TFD#8 was SolidFire Inc, another startup company focusing on selling entirely solid-state disk arrays. As you’d expect, they have their own niche and part of the market they are targeting with their all-flash drive product. So how do they compare to the competition and what’s their “unique selling point” compared to the likes of Violin Memory and Pure Storage?

Background

Solidfire was founded in 2009 by Dave Wright, who previously had created Jungledisk, a cloud backup provider. When Jungledisk was acquired by Rackspace, Dave had an epiphany around the way in which storage arrays were being used by cloud IaaS (Infrastructure as a Service) providers. This is the genesis of SolidFire and forms the basis for some of the specific features the SolidFire arrays offer.

The Offering

SolidFire have taken a slightly different approach to similar vendors in this space and chosen to deliver their product as a cluster of computing nodes. Each node contains processing, memory and disk in a fixed format, with scalability achieved by adding nodes in a clustered configuration. Performance is claimed to grow linearly across the cluster, as data is spread across all cluster nodes for both capacity and I/O load. This distribution mechanism provides for both consistent performance but also adds redundancy, with data replicated between nodes at the block level. A failure in a disk or node is handled automatically, re-establishing data redundancy.

SolidFire currently sell one model of storage node, the SF3010. This is a 1U “pizza box” rack-mount server with two Intel Xeon 2.4Ghz 6-core processors, 8GB of NVRAM (write cache) and 72GB of memory. Each node has ten 300GB 2.5″ SSD drives for a total of 3TB of raw storage. SolidFire have chosen the iSCSI protocol for front-end connectivity, with each node having two 10GbE connections. This choice of protocol is probably due to the clustered design; Fibre Channel isn’t easy to cluster without suitable multi-path software on the host servers. A single SolidFire cluster can scale from 3 to 100 nodes. Clearly a single node is not enough to run a resilient system, hence the recommendation for a minimum of three as the smallest configuration.

With a large amount of processing power per node, data entering a SolidFire cluster is de-duplicated and compressed inline. All LUNs are also thin provisioned. None of these features are user configurable and are all enabled by default. As a result, the 3TB raw per node translates to 12TB usable. We are seeing other SSD array vendors making similar claims that the usable capacity is more than the raw capacity due to data optimisation techniques. Designing these features into an array from the outset (especially with lots of CPU performance, memory and fast disk access) is something traditional vendors will struggle to emulate.

Secret Sauce

Every vendor needs to have their own “secret sauce” and SolidFire are no exception. It’s important to look at their target market, which is cloud-based service provision. This can mean both internal and external clouds, however the key message is that traditional provisioning methodologies don’t scale and don’t fit in automated environments. This is pretty easy to see this when looking at traditional storage management provisioning tools. SRM software is focused on interaction with the administrator, providing only point in time current views of storage and very few tools do concurrent provisioning well.

SolidFire have taken the approach of developing a REST-ful API for array management. This provides for all of the standard tasks of LUN creation, mapping and destruction with the ability to handle hundreds of API calls per second across a cluster. API integration is essential for any organisation looking to develop an automated cloud-syle provisioning process and this is an area where traditional array vendors simply can’t compete. API functionality can be integrated into existing processes and removes the need for large numbers of storage administrators – something that may be a worry to many, but we’re moving past the days of managing individual files and LUNs.

Another part of the “secret sauce” comes in managing I/O workload. Traditional arrays work on the assumption that all I/O needs to be delivered as quickly as possible and in the order in which it is received. There are some (but very few) arrays today that enable an administrator to prioritise workload. At best, all that is achieved is literally that – prioritisation rather than quality of service (QOS). I/O is still delivered as fast as possible, without regard for the service level needed for the I/O request. SolidFire have addresses the QOS issue by allowing individual LUNs to have minimum, maximum and burst levels of performance applied. This means LUNs created on day 1 of a new cluster deployment should receive exactly the same QOS and I/O performance when the cluster is fully loaded.

Deployment Scenarios

The ability to implement QOS features in cloud computing can’t be stressed highly enough. The first wave of IaaS (infrastructure as a service) enabled functional deployment – that is, proving workload could be moved to the cloud. The next wave will offer more QOS options other than simple processor and memory increments and storage will be one of those features. SolidFire arrays enables cloud providers to deliver differentiated levels of performance without having to deploy multiple tiers of storage. This is an important point. As soon as storage tiering is implemented, then efficiency drops, as there are always tiers that remain partially used. Block level tiering fixes some of these issues, but requires data to be moved around as part of performance re-balancing and still needs storage arrays to be monitored and assessed when adding additional storage. In addition, traditional arrays deliver I/O as quickly as possible, which can result in servers receiving more throughput than expected when an array is lightly loaded, but lower performance over time, which can be perceived by the end user as a performance problem.

The SolidFire solution will definitely see deployment in those organisations who have adopted a service-based approach to delivering computing services. With the API, QOS and node-scalable functionality, it is tailor-made for cloud deployments.

Summary

Solidfire started shipping in 2Q2011 with a $/GB price “similar” to that of traditional arrays. As with other vendors, the use of inline compression and compaction is being used to achieve an aggressive price point. Delivering for the cloud market is a smart move, as is the use of a scale-out node architecture that can grow as storage demand increases within an organisation. Cloud deployments use templated configurations, so the ability to configure and map LUNs via an API with no user intervention fits with automated orchestration requirements. I can see SolidFire arrays being widely used in many places over the next few years.

Additional Resources

The following video of Dave Wright was recorded at TFD#8.

Click here to view the embedded video.

Disclaimer: I attended TFD#8 as an invited blogger. My accommodation, some transportation and most meals were paid for. I was not compensated for my time, nor required to blog on any of the presentations. None of my blog entries, or other postings receive any pre-approval or viewings from vendors before publication.

IP Expo – 19 & 20 October 2011

admin — Mon, 17 Oct 2011 07:04:17 +0000

This week I’ll be attending IP Expo, which has probably become the UK’s biggest IT event. You can find more details out at the IP Expo Website, but if you’re quick, entry is free.

I’ll be there both days and I’m pretty busy with vendor briefings, but if you are going, drop me a note and we can meet up over a coffee (or beer in the evening). Remember, Steve Wozniak is doing the keynote on day 2, which should be poignant and interesting. See you there!

Solid State Arrays: Pure Storage Inc

admin — Sat, 15 Oct 2011 10:16:55 +0000

A clear focus of the recent Tech Field Day #8 event was the use of flash storage (or SSDs) for storage arrays or within hybrid solutions. Pure Storage offers an all flash storage array, which they say can be delivered at the same cost or less than traditional solutions. It’s a big claim, bearing in mind the cost per GB multiplier of flash over traditional HDDs is still pretty high. However delivering storage isn’t all about cost per GB and server workloads are changing, so perhaps we’re coming close to the point where all flash arrays are viable. This is a discussion that’s been had before with the likes of Violin Memory, so let’s dig a little deeper and see what Pure Storage has to offer.

Background

Pure Storage was founded in 2009 by John “Coz” Colgrove and John Haynes, who have their background in Veritas and Yahoo respectively. More details on their backgrounds can be found on the bios page of Pure’s website. Other members of the team have worked in companies such as Netapp, Decru and Sun, including Michael Cornwall, who was the lead flash designer at Apple for the iPod and iPhone. So far, the company has raised around $55m from investors, the most significant of which has to be Samsung, who provide all of the flash drives used in Pure Storage’s products. We’ll touch on the relative merits or disadvantages of this later.

The Offering

So what are Pure Storage offering? Well, it’s pretty simple; an all-flash storage array at a $/GB price that’s cheaper than traditional storage. Today that consists of two models, the FA-310 and the FA320. The FA-310 is a single controller with one storage shelf, providing up to 140,000 4K random write IOPS. The FA-320 doubles the storage capacity and increases write IOPS to 180,000.

Focusing on the FA-310, the controller is based on two 6-core Intel Xeon processors with 48GB of memory. Back-end connectivity is 6Gbs SAS and 40Gbs Infiniband, while front end connections are only based on Fibre Channel at this time (4x 8Gbs SFPs). Storage is provided by 22x 256GB MLC flash drives, giving a raw capacity of around 5.5TB. It’s not surprising that Fibre Channel is the only protocol available on the first models. FCoE doesn’t have the adoption rate and iSCSI wouldn’t suitable for the type of traffic this array can support. However the controller is detailed as having one spare expansion port, so we can speculate whether that is planned to be for Ethernet in the future.

The connectivity between the controller and disk is less than that offered at the front end. This may seem odd but it reflects on one of the key features of the Pure Storage arrays. Data entering the system is compressed and de-duplicated before being stored on disk, improving the overall efficiency of the array and reducing the volume of write I/O to physical media. The ability to perform data reduction before storing on media is the main way in which an acceptable price point can be met. This is something many other vendors are also doing as most customers are clearly fixated on the $/GB formula as the only way to measure acquisition cost. Pure quote a ratio of anything from 5-20x reduction and as anyone familiar with data reduction technologies will know, your mileage will vary depending on the type of data consumed.

Secret Sauce

Ultimately though, there has to be something that make Pure Storage stand out from the competition. During our Field Day visit, we were lucky enough to have a presentation from Coz, without slides, using just the whiteboard. He detailed what is probably the most important piece of Pure’s technology, and that’s the way they manage the SSDs themselves.

Solid state drives are fickle devices. Every write wears them out and much effort has been put into technologies (like wear levelling, write amplification) to extend their lifetime. MLC devices now have a much better reliability than they did a few years ago, allowing them to displace SLC in enterprise technology. Understanding how to manage SSDs is Pure Storage’s secret sauce. They work closely with the SSD manufacturers to understand the best ways to read and write from the devices in order to gain both maximum performance and maximum lifetime. At the presentation they even claimed never to have had an SSD failure, something that was met with surprise by the audience present!

Part of the SSD management involves the use of RAID-3D, technology which manages the RAID stripe distributions across the disks. RAID stripes are varied dynamically based on workload and the drive responses. This allows failing drives to be avoided, increasing their lifetime. It also means I/O response times can be made more predictable, avoiding random I/O spikes seen with individual SSDs as features like garbage collection kick in.

It makes sense to understand the best way to manage SSDs and having a relationship with the vendor of those devices certainly helps. My only concern is whether single supplier relationships are ever good, from a cost, reliability and supply perspective. Only time will tell.

Deployment Scenarios

So where would you deploy this kind of high performance array? I don’t think simply replacing your traditional storage with a Pure array is the right approach. One of the benefits of shared storage is that I/O demand consists of peaks and troughs, periods of high and low demand from many servers. This means it isn’t necessary to deliver 100% full I/O performance to all servers all of the time, but only to meet peak demand, which is considerably cheaper to achieve than meeting maximum demand.

This isn’t the Pure approach. Their arrays are capable of delivering 2000 IOPS for every TB of storage, even at 10:1 compression. It means that the server environment driving this storage needs to have high I/O requirements across every TB of data. Otherwise, the array is never running at peak efficiency. It could be said that if the $/GB cost is at a parity with traditional arrays, then should this matter? I think it does matter because there’s a perception that flash is an expensive technology and irrespective of the effective $/GB cost after data reduction, many customers will still focus on the raw storage and the cost of the device.

Summary

Pure Storage have done a great job in delivering a technology that brings solid state performance at an acceptable $/GB price. There are some key features (data reduction, SSD management) that make this technology really work. We have seen presentations of high I/O workload that can easily be managed by the Pure storage arrays, while continuing to deliver sub 1-millisecond responses. None of the big storage vendors have technology that delivers I/O bandwidth in a way companies such as Pure Storage can. All-flash versions of traditional arrays don’t have the added intelligence to manage SSD failures and peformance spikes. I can therefore see that very quickly one of the three letter storage companies will be looking to acquire Pure or one of their many competitors. For now, they need to focus on finding the right niche for their product, while educating customers in metrics other than $/GB.

Additional Resources

Pure were one of the companies at TFD#8 that were well organised in providing various pieces of media. I’ve included some of them here, including a link to the entire presentation from the day. There’s also a business-card sized set of instructions that Pure claim as their user manual. It’s a fun way of demonstrating how simple their technology can be.

Click here to view the embedded video.

SNW Europe 2011 – Powering the Cloud

admin — Fri, 14 Oct 2011 08:43:33 +0000

I’ve a busy few weeks planned for the end of October and November. One event I’ll be attending is SNW Europe 2011.

The event takes place in Frankfurt between 2-3 November 2011, with the strapline “Powering The Cloud” – you can find a link to the website here: http://poweringthecloud.com/home. There are actually three events taking place at the same time. SNW Europe, Datacenter Technologies and Virtualisation World. Can an event happen these days without mentioning virtualisation and cloud technologies?

Although the venue is small by comparison with some others, last year there was a good atmosphere in general, with some interesting sessions that possibly didn’t turn out as expected for the vendors in question. There’s also #storagebeers both evenings of the event as detailed here (https://www.facebook.com/groups/173383392686537/) although you’ll need to confirm your attendance.

This year I have no set agenda other than to perhaps find things that are new and interesting. Drop me a line or comment if you’re going!

Note: My attendance at SNW Europe is paid for by the organisers. This includes flights and hotel accommodation. I am not compensated for my time.