Review: Sun Storage 7000 Unified Storage System – Part I

]]> This is the third in a four-part series of posts on the Sun Storage 7000 USS storage arrays.  Previous posts in this series can be found here:

Review: Sun Storage 7000 Unified Storage System – Part I

Review: Sun Storage 7000 Unified Storage System – Part II

In this post, I’ll be discussing some of the analytics features of the 7000 arrays and how the rich data it offers provides customers with added insight into getting the best performance for their money.

Understanding Performance

To many people, storage array performance is a bit of a black art, usually performed by the skilled specialist or vendor and requiring lots of time and effort in planning layout, design and complicated measurement and calculation.  On legacy storage arrays this is possibly true, especially where data is tightly located to specific RAID groups or physical disks.  Some of the newer storage array technologies (and now features added to older arrays) allows for data to be dispersed across more physical devices and so to gain performance improvements from “wide-striping”.  However performances are achieved, there’s one thing for certain, you can’t improve your performance profile unless you can measure it.

USS Analytics

One of the key features of the USS systems is Analytics, which basically translates to rich statistical reporting on the status of the array.  Have a look at the graphic on the right here.  It shows a section of the dropdown list of metrics that can be selected on the USS for reporting purposes.  You can quickly see that reporting breaks down into the basic component functions of the array; CPU, memory, disk and in this instance cache, which relates to the operation of the ZFS filesystem.

There’s no doubting this is powerful stuff; USS Analytics lets you look at every component of your hardware and software stack, including protocol performance – in detail – both in real time and historically (historical data is kept pretty much forever).

OK, enough of what seems like marketing spin.  What counts is how analytics delivers for customers and how it adds value.  Well, from my perspective, performance tuning is all about obtaining the data required to make informed changes to the storage environment.  USS provides a rich source of performance information, in a simple format and most important – it’s free and bundled into the price.  In addition, the data is not being kept in a proprietary format and

can be accessed through CLI and XML interfaces.  This is extremely important for users as we move towards the age of commodity storage.  Devices that can’t fit into a standard management framework will be at a serious disadvantage as more and more organisations look to deploy storage for the lowest cost without compromising availability and performance.

Review: Sun Storage 7000 Unified Storage System – Part I

So in the first post in this series I discussed the USS and gave a basic overview of the hardware.  In [...] ]]> This is the second in a series of articles on Sun Microsystem’s Unified Storage System, also known as Amber Road.  Previous post(s):

Review: Sun Storage 7000 Unified Storage System – Part I

So in the first post in this series I discussed the USS and gave a basic overview of the hardware.  In this post I’ll discuss the disk components of the hardware in more detail and look at the use of flash (SSD) drives and ZFS to produce a commodity storage device.

Traditional storage arrays permit the configuration of multiple disk types within a single array.  This can range from solid state disks (SSDs), through to fast fibre channel drives and slower high capacity SATA drives.  USS operates a slightly different model – all drives in the USS array are high capacity SATA.  SSD drives are then used to ameliorate performance on read and write activity in combination with the ZFS file system, by using the SSDs for read caching and write logging.

How ZFS & SSD Are Used

OK, I’m not going to post a long diatribe about ZFS (although I may in the future), however it’s worth just having a look at the basic concepts in order to understand how ZFS impacts USS performance.  So, ZFS (originally called “Zettabyte File System”) is a high performance, high capacity filesystem introduced into Solaris about three years ago.  It is more resilient than UFS, not requiring filesystems checks after a system crash.  It also integrates the features of a standard filesystem and volume manager, pooling physical disks into groups from which filesystems can then be created.  ZFS supports RAID protection, including RAID-1 and RAID-Z, a proprietary implementation of RAID-5.  RAID-Z doesn’t suffer the same performance penalty as traditional RAID-5 as ZFS uses a Copy-on-Write (COW) methodology to write data into new locations rather than overwriting the original position.

ZFS uses two features (which relate to USS) to improve performance.  Firstly, disk reads are held in cache (called ARC  - Adaptive Replacement Cache).  Second, disk writes are journalled  (or logged) into the ZIL (ZFS Intent Log).  The ZIL provides resilience in the event of a system crash, however it also offers the opportunity for increased filesystem write performance.  Have a look at the graphic on the right, which is heavily used in the Sun documentation on USS.  This shows how traditional storage pools would be allocated with RAM and disk.  The USS model implements ARC for cached reads (which is stored in RAM), L2ARC, a level 2 ARC which extends ARC and is stored on read-biased SSDs and the ZIL, which is stored on write-biased SSDs.

L2ARC allows cache reads to be improved by creating an intermediate tier of read cache between disk and main memory.  ZIL improves writes by logging them to SSD and periodically flushing them to physical disk.  In the event of a system crash, integrity is still maintained as the ZIL is non-volatile.

In the USS, SATA drives are used in the main disk pool.  STEC SSD drives are used for the L2ARC and ZIL.  The model I reviewed had 36GB of ZIL cache, deployed as two 18GB SSD modules in standard disk enclosures.  The current implementation of USS only allows for a single disk pool, which means all data has to be protected with the same RAID level.  This is an annoying restriction, but I expect it will change in a future release as creating separate pools is simply a ZFS feature.

Why SSD and SATA?

It’s worth touching on why the USS is different to a traditional storage device.  In a typical general storage array there will be LUNs presented to hosts which are very active, some moderately active and some totally inactive.  If the LUN activity is plotted on a graph with the busiest LUNs on the left, the least active on the right and the Y-axis showing the degree of activity on each LUN in IOPS, the profile of a normal system will follow the “Long Tail” model.  This variation in activity is why savings can be made from operating a tiered model in a large storage array , placing LUNs on the appropriate tier of storage based on their activity level.

However, the trouble with taking I/O profile snapshots is that they’re just that – a snapshot.  They represent the I/O activity at that point in time.  Take a sample at another time of day or day of the week and another profile results.  This may show a very different set of busy LUNs compared to those highlighted previously.  One option is to average out the profiles over a suitable interval – say a day, a week or a month.  Whilst this will show on average the busiest LUNs, it will also mask any potential peaks in I/O demand as they will be averaged out over the period.  The shorter these peaks are, the less likely they will be noticed.

Deployment of tiering has one other problem and that is determining the amount of storage required in each tier.  It may well be that the ratios of each storage tier required changes over time as an array grows in size.  Perhaps the consumers of storage on the array realise that tier-1 storage is expensive and ask for more tier-2 or a new project comes along that needs a large volume of tier-0 SSD.  Typically, traditional arrays are inflexible at physically swapping tiers of storage on demand.

The USS provides one option to the Long Tail model.  By accepting all writes into SSD and destaging later to SATA, it ensures that high performance non-volatile storage is available at the time of the write and for multiple successive reads.  Fronting disk access with SSD ensures that high performance is dynamically provided to LUNs as it is needed.

Now it would be possible to compromise the SSD write cache by flooding a USS array with writes and this would be true for any array.  The question is at what point the USS would fail.   Unfortunately with my testing, I wasn’t able to generate sufficient  workload to overwhelm the 7210 I tested.  However I can say that in the testing I performed, the array coped easily with the workload I threw at it.  Clearly there’s still a requirement to manage the ratio of SSD to SATA based on the workload profile of the array.

Value Proposition

So what’s the value of using SATA and SSD in combination as the USS does?  There are a number:

• All data is stored on cheap, high capacity SATA drives, reducing the overall cost of the solution.
• I/O performance demands are managed by a small incremental cost in SDD.
• Variations in I/O workload performance is dynamically managed, removing the need to implement multiple storage tiers, significantly reducing management overhead.
• Array expansion is simplified – there’s no need to spend time planning how additional storage should be assigned to an array by tier.

Next time I’ll look at the analytics provided by the USS and how it allows detailed device reporting.