The VSP is the first enterprise array from Hitachi that offers support for 2.5″ drives.  We’ve seen these Small Form Factor (SFF) models for some time [...] ]]> This is a series of posts that cover the features of Hitachi’s new enterprise storage platform, the VSP (Virtual Storage Platform), also sold by HP as the P9500 array.

The VSP is the first enterprise array from Hitachi that offers support for 2.5″ drives.  We’ve seen these Small Form Factor (SFF) models for some time now and other manufacturers of modular storage arrays have chosen to use them.  Why is this move being made now by Hitachi and what can we expect from the market in general?

Drive Size

I’ve made many comments on 2.5″ drives over the years (here and here for example) and from time to time I’ve tracked the capacities of 2.5″ drives compared to their 3.5″ counterparts.  The tipping point has pretty much been reached, with 2.5″ 10K 600GB/450GB/300GB/146GB models available from both Seagate and Western Digital and 15K drives available up to 146GB from Seagate.  Whilst 15K 2.5″ drives have yet to catch up to their 3.5″ counterparts, 10K drives are clearly on a par (the largest 10K 3.5″ drives are still only 600GB) and consequently offer much higher volume density with lower power/cooling requirements.  As an example, a typical 600GB 10K 2.5″ drive will run at 8.32W compared to 10W for a 3.5″ equivalent; the 2.5″ drive will idle at 4.6W compared to 6.49W for its 3.5″ equivalent.  Although these numbers are small, scale them up across 1000 drives and you’re looking at a saving of nearly 2000W, or 30% irrespective of the number of drives.  Now cost is a factor; if the price per GB of 2.5″ and 3.5″ drives isn’t comparable, then their use may not stack up, however a quick street price check shows 600GB 2.5″ Savvio drives at £318 and 600GB 3.5″ Cheetah drives at £333, making the 2.5″ drive cheaper.

10K speed drives may not be suitable in all occasions and it’s true that the 15K 2.5″ drives still lag behind in capacity.  However the VSP also provides block-level tiering, enabling a much finer level of granularity in terms of active data placement.  This means bulk deployment of 15K drives isn’t as important; the high performance requirements can be met with 15K or SSD drives.  I’ll discuss more on block-level tiering in another post.

Drive Connectivity

Hitachi have made a decision with the VSP not to support fibre channel drives and all back-end drive connectivity is now SAS (Serial Attached SCSI).  This has facilitated the move to 2.5″ drives but also provides greater stability over traditional FCAL (Fibre Channel Arbitrated Loop) configurations.  SAS is a point-to-point protocol rather than loop-based which means failing drives cause less impact.  I’ve seen examples on FCAL arrays where the loss of a drive triggered a failure in the entire loop and 40 drives dropping out of their RAID set.  Although no data was lost, the configuration was severely compromised and all of the failing RAID groups required a rebuild when the problem was corrected.  With up to 2048 drives per array, back-end drive reliability is essential.

SAS has been available for some time on the Hitachi AMS models and other vendors have already moved their technology over.  In terms of bus speeds, SAS now supports 6Gb/s, with Fibre Channel languishing behind at 4Gb/s.  Remember that even though the drives can’t provide sustained transfers over around 130-140MB/s, the speed of the backplane is important as it enabled high burst speeds and affects the overall throughput writing to large numbers of drives.

Summary

Hitachi have moved the back-end architecture on to the next level.  The use of SAS and 2.5″ drives is a step over the competition (VMAX still uses FCAL and 3.5″ drives) and provides higher performance and reliability and reduced cost through space & power/cooling savings.  Although 2.5″ drives are slower speed, this is mitigated by the use of block-level tiering.  It will be interesting to see how long it takes other vendors to follow Hitachi’s lead.

I [...] ]]> They say that you learn the most when you make mistakes and things go wrong.  Well, last night I certainly must have learned a lot.  What started as a simple physical re-organisation of my hardware turned into a rebuild of my production VMware ESXi server – finishing at 1am.  Here’s what happened.

Failing Disk

I started by shutting down and moving my production ESXi Server out and back into the standard rack it occupies.  On power up, the server failed to reboot, claiming the boot disk was no longer present.  A quick check inside showed that the SAS connector on the boot disk had come loose, so I plugged it back in and tried again (Oh, SAS specification guys – bad design, no retainers on the plugs).  Unfortunately, the boot disk had somehow become corrupted and the server wouldn’t come up.  No problem, I thought, just repair using the installation media.  This is where things started to get complicated.

My ESXi server runs off a Seagate Savvio 2.5″ 15K 73GB drive, one of four Seagate generously loaned me last year for long term testing.  More on that another day.  The server has two disks installed, one of which has VMs on it.  During the repair process I wasn’t sure which disk was the O/S and which was data.  ESXi doesn’t help much, only indicating that both disks contained data in partitions, data that would be lost if I reinstalled. 

Lesson 1 – Make sure you know exactly how your hardware is configured, down to the SAS ports each drive is plugged into.

Actually having multiple drives of the same type is a pain.  So rather than risk data loss, I removed both drives and re-installed the ESXi O/S from a third Savvio drive.  All good.  Now I need to locate and import all my VMs, however some were on the removed Savvio disks.  This meant installing each disk independently and checking the contents to determine which contained VMs and which contained the broken O/S.

Lesson 2 – Wherever possible, place your VMs on disks separate from the server itself.

Yes, I do have most of my VMs on my Iomega ix4-200d, but, rather crucially, not my Windows 2008 AD Server, which needed to be moved from internal disk to the ix4 before I continued (schoolboy error there).  The AD server was rather important for accessing my, ahem, ix4, which is configured to validate logins using AD.  This creates a bit of a circular reference which could have been a disaster.

Lesson 3 – Place your Windows domain controller on a physical server, or have another independent backup elsewhere.

Having a physical server just for AD control isn’t part of my total virtualisation plan, so I’m looking at whether I can host a backup controller with Amazon AWS and use VPN to secure it into my private network.  This way, if I ever have an issue, I can still authenticate.  The issue of course is cost, which may make a dedicated server the cheaper option.

So, by 1am everything was back up and running.  Did I learn anything else?  Well yes…

Lesson 4 – after 22 years in IT, I should remember that adequate documentation and a DR plan are crucial.  In fact, in a virtualised environment, they are essential due to the concentration of risk placing all systems on a single server causes.

So what next for my virtual infrastructure?  I have a few changes planned; I’ll create a backup ESXi server that can import and run the VMs in the event of a future server failure.  I will also be investigating AWS with Windows 2008 and VPN to create a backup domain controller and see if I can continue to work if both server’s hardware failed.

That leaves one Single Point of Failure… my ix4-200d.  Anyone want to donate me a spare one?

I [...] ]]> They say that you learn the most when you make mistakes and things go wrong.  Well, last night I certainly must have learned a lot.  What started as a simple physical re-organisation of my hardware turned into a rebuild of my production VMware ESXi server – finishing at 1am.  Here’s what happened.

Failing Disk

I started by shutting down and moving my production ESXi Server out and back into the standard rack it occupies.  On power up, the server failed to reboot, claiming the boot disk was no longer present.  A quick check inside showed that the SAS connector on the boot disk had come loose, so I plugged it back in and tried again (Oh, SAS specification guys – bad design, no retainers on the plugs).  Unfortunately, the boot disk had somehow become corrupted and the server wouldn’t come up.  No problem, I thought, just repair using the installation media.  This is where things started to get complicated.

My ESXi server runs off a Seagate Savvio 2.5″ 15K 73GB drive, one of four Seagate generously loaned me last year for long term testing.  More on that another day.  The server has two disks installed, one of which has VMs on it.  During the repair process I wasn’t sure which disk was the O/S and which was data.  ESXi doesn’t help much, only indicating that both disks contained data in partitions, data that would be lost if I reinstalled.

Lesson 1 – Make sure you know exactly how your hardware is configured, down to the SAS ports each drive is plugged into.

Actually having multiple drives of the same type is a pain.  So rather than risk data loss, I removed both drives and re-installed the ESXi O/S from a third Savvio drive.  All good.  Now I need to locate and import all my VMs, however some were on the removed Savvio disks.  This meant installing each disk independently and checking the contents to determine which contained VMs and which contained the broken O/S.

Lesson 2 – Wherever possible, place your VMs on disks separate from the server itself.

Yes, I do have most of my VMs on my Iomega ix4-200d, but, rather crucially, not my Windows 2008 AD Server, which needed to be moved from internal disk to the ix4 before I continued (schoolboy error there).  The AD server was rather important for accessing my, ahem, ix4, which is configured to validate logins using AD.  This creates a bit of a circular reference which could have been a disaster.

Lesson 3 – Place your Windows domain controller on a physical server, or have another independent backup elsewhere.

Having a physical server just for AD control isn’t part of my total virtualisation plan, so I’m looking at whether I can host a backup controller with Amazon AWS and use VPN to secure it into my private network.  This way, if I ever have an issue, I can still authenticate.  The issue of course is cost, which may make a dedicated server the cheaper option.

So, by 1am everything was back up and running.  Did I learn anything else?  Well yes…

Lesson 4 – after 22 years in IT, I should remember that adequate documentation and a DR plan are crucial.  In fact, in a virtualised environment, they are essential due to the concentration of risk placing all systems on a single server causes.

So what next for my virtual infrastructure?  I have a few changes planned; I’ll create a backup ESXi server that can import and run the VMs in the event of a future server failure.  I will also be investigating AWS with Windows 2008 and VPN to create a backup domain controller and see if I can continue to work if both server’s hardware failed.

That leaves one Single Point of Failure… my ix4-200d.  Anyone want to donate me a spare one?

Demand at Seagate is down and consolidation of the industry is expected. However as recently as March last year EMC was telling us how storage growth just keeps on spiralling upwards.

So what’s happening? Are we becoming inherently more efficient [...] ]]> Eric Savitz over at Tech Trader has an interesting article today.

Demand at Seagate is down and consolidation of the industry is expected. However as recently as March last year EMC was telling us how storage growth just keeps on spiralling upwards.

So what’s happening? Are we becoming inherently more efficient at storing our data all of a sudden, now that a credit crunch is upon us? Somehow I don’t think so.

Demand ebbs and flows as finances dictate the ability to purchase new equipment, but growth remains steady. Technology is replaced constantly but just like you or I might hold on to our car for another year or so before replacement, so will IT departments, preferring to pay maintenance on existing kit rather than rip and replace to the latest and greatest. I can see two consequences from this;

• More time and effort will need to be paid to using current resources more efficiently.
• Migration to new hardware will need to be even more slick and quick to reduce the overhead of migration wastage.

I’ll discuss these subjects in more detail this week.

The capacity increase is overdue to keep up with the [...] ]]>

Meantime, last December I posted on the subject of 2.5″ drives in Enterprise arrays and created this spreadsheet comparing different models. The reason for creating the sheet was to see if the physical density of 2.5″ drives would exceed that of traditional 3.5″ models. At the time, the best 2.5″ drive offered 0.702GB/cm3 compared to a slightly better 0.796GB/cm3 for the 3.5″ equivalent (73GB versus 300GB drives respectively).

A quick check on their website shows precious little information. In fact [...] ]]> Over 18 months ago, I discussed in a post (here) the Hybrid Storage Alliance who were looking to increase the performance of hard drives by adding large amounts of flash to each device. But where are they now?

A quick check on their website shows precious little information. In fact what’s on offer seems to point to using hybrids for just laptops and Vista. Unless they’re working on new products, it seems to me that the manufacturers of hard drives have missed a trick.

EMC have made a clear stance to move towards flash (with their EFDs) but in reality they’re targeted as specific applications requiring super-fast performance. In any case, the cost will make them prohibitive for most customers (initially, at least). Drive speeds are fairly much plateaued at 15K (although some manufacturers are discussing 20K spin speeds) as the power/cooling issue is obviously a problem when going at faster revolutions.

Taking these factors into consideration isn’t there a gap in the market for Enterprise Hybrid Drives? I’d like to stake a claim for the term EHD at this point before anyone else does

It seems to me like a logical step; manufacturers such as Seagate won’t want to give up their dominance to flash in a hurry and in any case flash is too expensive for most usages.

Perhaps there are issues getting EHDs to work; you wouldn’t want to spin them down (although you might accept them spinning slower); maybe there are problems with effectively filling and destaging the flash cache on the drives which requires more thought from the array manufacturers. Either way I’m speculating but it seems there’s an opportunity there for the taking.

Remember, Enterprise Hybrid Drives, heard here first!!

Obviously the first question is to determine what the data profile is, however if it isn’t known, then assume the I/O will be 100% random. If all the I/O is random, then each I/O [...] ]]> A question I get asked occasionally is; “How many IOPS can my RAID group sustain?” in relation to Enterprise class arrays.

Obviously the first question is to determine what the data profile is, however if it isn’t known, then assume the I/O will be 100% random. If all the I/O is random, then each I/O request will require a seek (move the head to the right cylinder on the disk) and the disk to rotate to the start of the area to read (latency) which for 15K drives is 2ms. Taking the latest Seagate Cheetah 15K fibre channel drives, each drive has an identical seek time of 3.4ms for reads. This is a total time of 5.4ms, or 185 IOPS (1000/5.4). The same calculation for a Seagate SATA drive gives a worst case throughput of 104 IOPS, approximately half the capacity of the fibre channel drive.

For a RAID group of RAID-5 3+1 fibre channel drives, data will be spread across all 4 drives, so this RAID group has a potential worst case I/O throughput of 740 IOPS.

Clearly this is a “rule of thumb” as in practical terms, not every I/O will be completely random and incur the seek/latency penalties. Also, enterprise arrays have cache (the drives themselves have cache) and plenty of clever algorithms to mask the issues of the moving technology.

There are also plenty of other points of contention within the host->array stack which makes this whole subject more complicated, however, when comparing different drive speeds, calculating a worst case scenario gives a good indication of how differing drives will perform.

Incidentally, as I just mentioned, the latest Seagate 15K drives (146GB, 300GB and 460GB) all have the same performance characteristics, so tiering based on drive size isn’t that useful. The only exception to this is when a high I/O throughput is required. With smaller drives, data has to be spread across more spindles, increasing the available bandwidth. That’s why I think tiering should be done on drive speed not size…

I contacted HGST and Seagate to get some additional background. Here are their responses, slightly edited to correct any spelling mistakes but otherwise intact.

Seagate

(a) when will the technology be [...] ]]> Last week I posted about wasted hard drives, removed from arrays and crushed to prevent the leak of sensitive data.

I contacted HGST and Seagate to get some additional background. Here are their responses, slightly edited to correct any spelling mistakes but otherwise intact.

Seagate

(a) when will the technology be deployed in Enterprise FC drives? Our OEMs are currently developing with the Cheetah 15K.6 FDE, a drive that Seagate has already in production.

(b) is the technology proprietary to Seagate? – No, this will becompliant with the Trusted Computing Group’s spec. All hard drive vendors are participating in this Trusted Computing Group and we expect that they will have self-encrypting drives that will be inter-operable with ours.

(c) is DriveTrust accepted by the US Government and other similar organisations as secure enough to treat a drive as “wiped” if the encryption keys are removed? Endorsement from National Security Agency (NSA) has already been received for the 1st Self-Encrypting Drive Model-the Momentus(r) 5400 FDE hard drive, for protection of information in computers deployed by U.S. government agencies and contractors for national security purposes.

(d) are any of the “big” manufacturers (EMC/HDS/IBM) looking to deploy DriveTrust enabled drives in storage arrays? IBM and LSI have both publicly announced that they will do so. Note that Hitachi has also just announced a self-encrypting drive, the Deskstar E7K1000, a drive designed for business critical storage systems.

(e) Where do the drives go when they’re wiped for final disposal? Extra shipping is involved to ship a drive to a special data destruction service facility, where it can be degaussed or shredded, and then the drive must be shipped to [be] environmentally disposed of. Alternatively, a drive may be over written, a process that takes hours and hours, using energy and tying up system resources, and then may be re-purposed.

HGST

My name is Masaru Masuda, working on product planning for Hitachi GST. Let me try to answer your question. Like Raj mentioned below, we have already supported bulk encryption feature for 2.5″ and 3.5″and will support it to Enterprise product next year. With the bulk encryption feature, user data on the HDD media is automatically and always encrypted by the SoC inside [the] HDD. The security feature has two basic functions. One is active protection of data (encryption with secret key) and secure erase of the drive by deleting the encryption key for repurposing or disposal. As you pointed out, Standardization is a key for security. Therefore, a non profit security organization called TCG (Trusted Computing Group) was formed as described in the page 5 and 6 of the attached package. We have been very actively involved in the activities of TCG and plan to pick up security feature based on TCG standards which will be implemented from next year.The security market is still small but it has been growing steadily due to the data security concern and also as a fast and cheap solution for repurposing of drives in Server applications or disposal of failed drives. Also we have had a recycling process for drives failed in the internal testing and for drives returned from the field.

Thanks to both companies for their responses.

So it seems to me that in the future there will be no excuse for scrapping drives. I think the retirement process for HDDs should form part of the “green measurement” of storage.

I’ve trawled the [...] ]]> Seagate announced yesterday version 11 of their Barracuda hard drive range, to be released next month (August 2008) with a maximum capacity of 1.5TB. The news link has all the speeds and feeds if you’re interested in how they have achieved this remarkable milestone.

I’ve trawled the ‘net to plot the release of previous versions of the drive and their capacities at the time. Here’s a graph of the releases I could find, going back to 2002. Trending the growth (totally un-scientifically of course), then we can expect to see 2TB drives by December 2008, 3TB drives by November 2009 and 4TB drives by June 2010. This may be a little optimistic as the trending is skewed slightly by the recent advances perpendicular recording has brought to capacity growth, but maybe not, as my recent post on Hitachi 5TB drives shows.