The Cyclical Nature of Storage

By Chris M Evans On 7 July, 2012 · 7 Comments · In Opinion

Perhaps I should have entitled this post “The Cyclical Nature of IT” because the same principles apply. Over time, storage moves back and forth between fully dispersed and fully centralised like a sine wave on an oscilloscope.

The Evidence

Look at the evidence. In the 1950′s and 1960′s there was no choice but to be centralised. All we had was mainframe and dumb disk drives. Storage needed a controller the size of a fridge to work and so was never going to be distributed to the masses any time soon.

Then in the late 1970s, in comes Al Shugart and Seagate, giving us the first drive in a format we recognise today – the ST506. As soon as that point is reached, we can now deploy our own servers with dedicated storage and so we had the distributed storage sprawl of the early 1990′s.

This quickly evolved into SAN storage, which brought us back to that centralised storage model. Since then, we’re starting to see the centralised model fall apart as storage once again goes distributed. What’s the evidence for this?

Microsoft now recommends DAS for Exchange deployments
PCIe SSD cards are moving storage back into the data centre
Bespoke storage appliances (Tintri, Nutanix) are breaking the standard deployment rules
The days of the “single large storage array” are over – SSD arrays, unified storage, dedicated NAS, object, all exist

Of course the de-centralisation of storage is done for good reasons, whether that’s cost, increased performance or otherwise gaining business advantage.

The Architects View

No doubt, we will see storage diverge further as more bespoke solutions come to the market. It’s then easy to predict that storage will once again centralise. The skill will be in making the prediction on why and when this will happen. Anyone like to place a bet?

Tagged with: centralisation • SAN • Storage

-Tom

I tend to disagree with the argument that storage is being decentralized. First, cloud storage – growing impressively – is centralized storage through and through. Regarding PCIe storage, in the long-term, it will serve the role of cache, not storage, as evidenced by the approaches now emerging to mate PCIe with caching software. But while storage will remain predominantly centralized, multiple technologies and vendors may participate in the centralized “pool”. This is the result of virtualization deployments making it simpler to combine vendors and provide data management services outside of the array while provide seamless high availability and migration/movement capabilities.

http://thestoragearchitect.com/ Chris M Evans

Tom, perhaps the next centralisation will be Cloud-based. The problem with PCIe SSD as a cache is that it requires some application changes to make work. Yes there are caching applications out there, but the big challenge for PCIe SSD to survive is whether applications will be written specifically for it. I think there are too many variables in play to predict specifically what will happen, but it will certainly be interesting to watch.

Chris

Attila NIckl

You can also add:
- ”AlwaysOn Availability Groups” for SQL Server 2012 deployments
- New Windows Server 2012 virtualization and storage features like “Shared Nothing Live Migration” and “Storage Spaces”
Adam Bane

I think we’re already seeing the re-centralization of storage for certain types of data. S3 has already crossed the trillion-object mark and is believed on track to exceed an exabyte this year. Object storage systems (S3 and its clones, OpenStack Swift, and other proprietary platforms) provide a highly cost effective way to scale centralized storage. API access and the ability to associate metadata directly with objects reduces application development costs and easily allows multiple applications to access and utilize the same data without replicating or migrating it for each individual use.

While object stores certainly don’t eliminate the need for high-speed SAN and NAS environments, as more applications evolve to leverage object store APIs directly, we’ll continue to see the migration to centralized storage for the vast amounts of unstructured data that we’re generating.

http://thestoragearchitect.com/ Chris M Evans

Adam, I agree that cloud does offer centralisation options. Not sure we’re there yet and I don’t think cloud will enable centralisation of eveything, but it does demonstrate the process.

Chris