In this post we’ll discuss the logical configuration, connectivity and protocols available on the Compellent Storage Center array, including the way disks are grouped and LUNs are created from the underlying [...] ]]> This is a series of posts reviewing the Compellent Storage Center SAN Array.  Previous Posts:

• Review: Compellent Storage Center – Part I

In this post we’ll discuss the logical configuration, connectivity and protocols available on the Compellent Storage Center array, including the way disks are grouped and LUNs are created from the underlying storage.

Where’s My LUN?

Dynamic Block Archiecture

The first thing that should be noted as we dive into the detail on how the Compellent array stores data is that it does not operate like traditional storage arrays with disks in fixed RAID groups and LUN configurations, but uses the previously mentioned Dynamic Block Architecture.  RAID 10 RAID-10 DM (Dual Mirror), RAID-5 and RAID-6 configurations are supported (including both RAID-5 with 5 or 9 drives in a RAID set and RAID-6 with 6 or 10 drives), however that’s where the similarity with traditional arrays ends.  The underlying physical disks are actually simply grouped together to provide raw disk capacity and LUNs are configured from that storage.  RAID is applied to each individual block of a LUN and this can change over the lifetime of that block of data.  At the outset this may seem like a complicated design but in reality it isn’t.  By breaking down a LUN to the block level and then applying protection and performance criteria, Compellent can achieve higher performance from a system using less cache and crucially less high performance drives. Let’s start at the basic disk level and work up to define how the Compellent system works.

Disk Pools

Physical disks are classified by their rotational speeds, which effectively groups them by performance.  In the review hardware, drives were classified as SLCSSD (for STEC SSD drives), 15K (the fibre channel drives) and 7K (the SAS drives).  By default all disks are added to a single group (or folder) called “Assigned”.  Disks that are not in use are assigned to a dummy folder called “Unassigned” from where they can be added to a new or existing disk folder.  Compellent recommend keeping a single disk folder, as spares must be assigned within each group. Having multiple folders would both waste spares from a capacity perspective and reduce performance as I/O would be spread over less spindles.  Of course it is possible to create separate groups if you wish.

As part of the disk folder definition, either single or dual parity must be specified for each tier.  Screenshots 2 & 3 show the setup of the default “Assigned” group and a second “New Disk Folder 1″.  There are two other things to say about disk folders.  Firstly disks can be removed from a folder.  This requires “evacuating the disk” which can be achieved by moving it to another dummy folder.  If the disk contains no data, it can simply be removed from the folder.  Second, as disks are added to a folder, there’s the risk of RAID imbalance, with all of the data existing on the disks already in the disk  folder.  Therefore as disks are added, the RAID configuration can be rebalanced to obtain optimum use of all spindles.

Storage Profiles

The concept of Storage Profiles is where the Compellent Storage Center “secret sauce” is to be found.    These determine how the system writes data to disk for each LUN (or volume as they are known in Compellent terms) and how data ages over time – a feature called Data Progression.  Let’s look first at the profiles.

For each volume/LUN in an array, the Storage Profile determines how data is written to disk.  Storage profiles have two components, specification of where data should be written and specification on where Replay data should be located.  It’s worth taking a moment to understand what Replays are as I’ve yet to mention them.  Replays are essentially snapshots, used to return a volume to a previous point in time.  By their nature, Replay data blocks are only ever used for reads as all writes made after a snapshot/replay is taken will be written to a new location in order to preserve the replay for a potential future restore.  Replay blocks are therefore not part of the active write set of data being written to a LUN and don’t always need to reside on high performance storage; if they are being read frequently then they will reside in cache.  Storage Profiles allow the administrator to indicate what should happen to both writable blocks and replay blocks for a volume.  A high performance LUN could, for example, have its writable data on tier 1 storage with RAID-10 configuration and have replays on RAID-5 SAS.  A medium performance volume could have writable data on tier 2 15K Fibre Channel and replay data on SATA.

The use of Storage Profiles provides some very important benefits in optimising the performance of the disks in a Storage Center array.  They allow on a LUN basis, the exact performance criteria to be specified.  In addition, only active data is retained on the highest performance storage with inactive data moved off to lower performing (and lower cost) devices.  As RAID is established at the block level, this means a granularity on writes of 2MB in a standard configuration.  If required, volumes can be created using 512KB blocks where writes are small.

Data Progression

Replays and Data Progression

Although I’ve touched on the subject of classifying data into active writes and replays, I haven’t explained the actual mechanism in which data moves between these groups.  There are two methods by which data is migrated between tiers of storage; via Data Instant Replay and through Data Progression.  Replays as we have discussed are Point-In-Time snapshots of volumes.  When a replay is taken, all of the pages comprising a volume are frozen.  Subsequent writes to the volume are made to new blocks on storage.  This preserves the data at the point the Replay was taken and also quite helpfully allows the blocks that are being actively written to be distinguished from those which are inactive.  The Replay blocks can then be moved to a lower tier of storage.  Compellent recommend that every volume has a Replay taken on a daily basis.

Data Progression uses a similar technique to move blocks of data that are less frequently used, down to lower tiers of storage over time.  Initially all writes are made to the highest tier of storage and over time, migrated to lower tiers based on frequency of access.  This occurs at the block level and means Storage Center arrays can be configured with the optimal mix of different drive types.  For instance, if more performance is required, SSD could be added; if more capacity is required, SATA can be added.

In summary, the Compellent design enables data placement to optimised to the block level, with less frequently accessed data moved to lower tiers of storage.  In normal circumstances the default settings can be used but if specific high performance volumes are needed, this can be accommodated too.

Protocol Support

One final topic, as this is becoming a long post.  Storage Center supports both Fibre Channel and iSCSI connectivity.  Unusually for storage arrays it allows both protocols to be mixed to a single volume at the same time; so I/O can be actively using both fibre channel and iSCSI.  If you have the right version of switch firmware, Fibre Channel also supports NPIV, which enables the creation of virtual ports on physical ports.  I hope to go over this in a future post.

In the next post I’ll discuss performance and some of the other miscellaneous features such as replication and clustering.

Hot Spots

EMC’s [...] ]]> So, here’s my rash statement from Twitter last night: “If FAST isn’t free, I don’t want it!  All it’s doing is automating process I could script/do manually”.  It’s a bold statement, I know, so is FAST really offering something better than what could be achieved today using EMC’s Symmetrix Optimizer?

Hot Spots

EMC’s Symmetrix architecture (18 years old and counting, I believe) uses the concept of disk hypers to present LUNs.  Each physical disk is carved into a number of slices, which are then recombined to create LUNs to present to a host.  A mirrored (RAID-1) LUN uses two hypers, a RAID-5 (3+1) LUN uses 4.  EMC ensure general performance by setting standards on how LUNs are created from hypers and that’s reflected in a “binfile” layout.  However despite this sensible planning, it is possible (especially as hard drives are now much larger and contain many more hypers) that two hypers on a single physical disk could be highly active and so contend against each other – in other words “hot spots” on disk.

Optimizer helps alleviate the issue of hot spots by exchanging the high I/O hypers with low I/O ones, distributing busy LUNs across more physical spindles.  This is classic load balancing where resources are distributed across the available infrastructure in order to obtain better overall generic performance.  EMC have now rebranded Optimizer as part of Ionix for Storage Resource Managment, but it’s still effectively the same product.  Hyper swaps can be managed automatically, based on historical performance data.  They can also be user-defined – a manual swap at the users request.

Although tedious (and not as well automated as Hitachi’s HiCommand Tiered Storage Manager), in theory Optimizer could be used to manually move workload between storage tiers.  In fact, Optimizer is already aware of a tiered storage infrastructure.  Here’s a quote directly from the ControlCenter 6.1 manual:

“Optimizer is also aware of physical drives that operate at different speeds, as well as location of the data on the physical media, which influences the I/O rate. This information is used when determining which logical devices to move.”

So with a little bit of knowledge on the layout of data on a Symmetrix array, it would be possible today to use Optimizer to perform LUN-based FAST.

Load-Balancing Versus Policy

Unfortunately, simple load-balancing of I/O across a storage array doesn’t offer what should be seen as the next generation of storage tiering.  Where Storage Tiering 1.0 was about offering multiple layers of storage within the same physical infrastructure and manually placing or moving LUNs to the appropriate tier, Storage Tiering 2.0 will be about establishing policies that determine more service-based measurements of the performance and availability customers receive. 

A policy-based approach would allow rules to be established on how data at the application layer moves between tiers.  This is a critical distinction from the load-balancing  methodology earlier described.  As an example, where an application was known to require higher performance at a certain time of day or day of the week, data could be moved proactively to a faster tier of storage, returning later once the high I/O workload had completed.  Whilst achievable using Optimizer, there’s no doubt the process of application migration would be tedious and time consuming.  I expect the v1.0 implementation of FAST will simply package up Optimizer into a tool that automates the migration of related data between tiers.  Don’t forget, other vendors have been offering this feature for some time – for example Hitachi and Tiered Storage Manager.

Increasing Granularity

Now LUN-based migration has its benefits.  Where large numbers of disks exist in an infrastructure, application data can be placed or moved to the most appropriate location as required.  However with the introduction of solid state disks (SSDs), a more granular approach is needed as the number of SSDs deployed in an array is likely to be low due to their excessive cost.  Moving an entire application (or even LUN) to SSD will be undesirable unless that application can take full use of the SSD hardware.  There are very few, if any, applications that require high-intensity read/write activity from every piece of application data all the time.

Block-level tiering offers a higher level of granularity to the placement of data.  A LUN can be split into blocks and placed across multiple layers of storage technology including traditional HDDs and faster SSDs.  Selective placement will ensure the more efficient use of expensive SSD media by placing only the highly active data onto it.

All of a sudden with increased granularity we’re back to Storage Tiering 1.0 where data is being placed on faster technology purely based on increasing overall system performance.  This is a feature Compellent have been offering for some time.  Data is migrated up or down the tier hierarchy on a daily basis, subject to performance figures over a 12-day period.  This level of granular performance management is possible because data is stored in a block-based structure.  Unfortunately for EMC, the hyper design legacy represents a technical challenge in making FAST version 2 a reality. 

Patent Rights

As just mentioned, Compellent already offer block-based data migration in their products.  At a recent dinner in London with the Compellent team, they highlighted their strong position in the market, protected by patents covering block-level data migration between tiers.  You can find the filed patent here.  Compellent use the term “Data Progression” to describe how blocks are moved between tiers based on I/O activity.  As I/O activity is monitored over time, it is possible to determine the most appropriate tier of storage to use when expanding capacity.  Typically these are lower tier SATA drives, as initial performance requirements are usually over-estimated.  This metholodogy is very much Storage Tiering 1.0 discussed earlier.

Compellent aren’t the only people claiming rights to block-level tiering within a storage array.  I’ve also found the following patent application from IBM, filed by Barry Whyte, Steve Legg and others.  If IBM and Compellent both claim to have invented the FAST concept, how does that position EMC?  Do they have an earlier patent which trumps these two?

Summary

Storage Tiering 1.0 provides performance management of storage arrays.  Storage Tiering 2.0 extends this to offer policy-driven optimisation offerings.  Both of these technologies are available today from existing vendors in one format or another.  EMC will simply be playing catchup with these vendors once FAST 1 & FAST 2 are released.  I’d like to be surprised and see EMC offer something the competition currently don’t.  I’m not holding my breath…