The Storage Architect » iSCSI

Managing iSCSI on Windows 2008 Core with Hyper-V

Chris M Evans — Fri, 04 Feb 2011 16:37:48 +0000

I’ve been spending some time recently working with Microsoft’s Hyper-V virtualisation platform and one of the major hassles I’ve experienced is having to use the command line rather than GUI for certain functions. If you’re not aware, Windows 2008 Core is a stripped down version of a standard Windows 2008 Server deployment and so has no GUI interface. That makes administering features such as iSCSI a little more challenging as this isn’t one of the features of Server Management.

Image 1 - iSCSI GUI

If you use iSCSI in your Windows environment, you will be familiar with the standard iSCSI GUI. This GUI makes administering iSCSI incredibly simple (with functions like Quick Connect) or of course you can use the more advanced features that allow the configuration of security and other features. Image 1 shows an example of the iSCSI Initiator page from one of my Hyper-V test machines. You can just about see that some of the connections are established and some are inactive.

Of course there is no GUI to use in 2008 Core, so the fallback is to use the iscsicli.exe command. You can find a description of the iscsicli commands using the following link from Microsoft: iSCSI Users Guide. This Word document provides details on how to create, display and delete iSCSI connections from the command line. For example to list target devices, use the command “iscsicli listtargets” as shown in Image 2.

Image 2 - iscsicli

While these commands are good and useful, there are two problems; firstly they require logging onto the server to run them (I want to run them centrally) and secondly, the available commands don’t give me exactly what I want to know. To get the data I want, I’ve utilised WMI (Windows Management Instrumentation), which provides a huge amount of information on a running Windows system. The iSCSI details are all stored in classes that start MSiSCSI and MSiSCSIInitiator. I’ve created two Powershell scripts, called iSCSIListSessions and iSCSIListDevices. The first lists active and pending iSCSI sessions, the second is a bit more useful and lists sessions and the associated devices.

Each script requires the host name to be specified as a command line parameter. See Images 3 & 4 for example output. I’ve also attached both scripts to this post for download if anyone is interested in using them. Note: You use these scripts at your own risk. The scripts are pretty basic and assume the userid you’re logged in with is the one for accessing the remote server, so you need WMI permission on your logged in userid for this to work. If I get time, I may produce something more flexible that would allow credentials to be overridden.

Image 3 - iSCSIListDevices

Image 4 - iSCSIListSessions

iSCSIListDevices.ps1

iSCSIListSessions.ps1

Make sure you rename the file extension from .txt to .ps1 before using the scripts.

Hardware Review: Promise SmartStor NS4600 – Part III

Chris M Evans — Mon, 09 Aug 2010 20:42:20 +0000

This is a series of posts on the Promise SmartStor NS4600 home storage server. Previous posts:

In this post, we will discuss file layout, formats and protocols available on the NS4600. The previous post (above) discussed how RAID is constructed across physical disks. Multiple volumes can be constructed from the disks available in the system (subject to a disk being dedicated to only one volume). Above this layer sits the file system and logical iSCSI devices.

Filesystem Status

The first screen shot shows the normal status of a file system. In this example there are two drives paired together in a RAID-1 mirror, providing approximately 1675GB of storage space. At present only around 1442GB of space as been allocated to the file system, with some 187GB of free capacity still available. At first it may not seem obvious why all the available space shouldn’t be allocated to the file system, however the answer is simple; standard NAS file space and iSCSI LUNs sit beside each other together on the RAID volume. Therefore as a file system is created, it can be sized as required, allowing future expansion as either additional file space or as iSCSI LUNs. The creation of the iSCSI LUN is shown in subsequent screen shots, highlighting the initial available 187GB of space, followed by the creation of two 50GB iSCSI LUNs, reducing the available space to 87GB.

Mixing data types on the same RAID set would not necessarily be best practice on a medium-tier or enterprise-class array; fortunately the ability to create multiple volumes enables some disks to be dedicated to file and others to block-level access, simply by creating multiple volumes. Of course the main restriction is only having 4 drives to play with, however the underlying architecture enables multiple configurations to be created and potentially in the

future, should Promise choose to create larger devices, would offer the fundamentals for sensible data segregation. In any event, for single CPU, single NIC devices like the NS4600, high performance isn’t likely to be the main purchasing consideration and mixing file and block data on the same RAID group shouldn’t pose a problem.

Protocols

For file data, the NS4600 provides Windows, Mac, FTP and Unix connectivity. See screenshot 9 in the gallery at the end of this post. Protocols can be turned on/off system-wide or specified for each file share. For Windows, the server is able to connect to an Active Directory domain. This may not be everyone’s first choice but in a small office, centralising security is an essential requirement – I always look for Active Directory support as it simplifies my home lab setup. Mac settings are pretty simple; on or off, with the ability to issue a message to connecting systems. FTP is similarly simply specified and for Unix/Linux connectivity, the NS4600 can be connected to a NIS domain. It also seems possible to connect a share to both a NIS domain and AD at the same time. Now, unfortunately I don’t run NIS so couldn’t test this; however specifying two security domains does throw up issues of consistency and questions around which security model “wins” in the event of a conflict. There doesn’t appear to be any way to specify userid translation as there is in Data ONTAP for example.

Creating shares and using onboard security is straightforward. The screenshots in the gallery at the end of the post highlight how this is achieved. File shares can be created under the root of a volume and assigned permissions based on internally connected users, or on imported users from external sources like AD.

Ok, now we have the file system basics out of the way, the next two (and final posts will be more interesting). Next I’ll discuss backups and replication and the final post will look at the NS4600 as an application server and the other features it offers.

Hardware Review: Drobo Elite – Part I

Chris M Evans — Wed, 02 Jun 2010 07:00:59 +0000

Drobo Elite - Rear View

This post is the first in a series covering the DroboElite SMB storage device from Data Robotics Inc.

The DroboElite is the most fully featured storage array in the Data Robotics range of devices. Regular readers will know I’ve posted frequently on these devices, from the first Drobo “Classic” to the DroboPro, both of which I own and have purchased myself. Data Robotics have kindly loaned me a DroboElite for this series of posts. As you will see, the top of the range model offers all the features the other models provide, plus the benefits of multi-user support.

Background

The original Data Robotics storage device was what is now referred to as the Drobo “Classic”. This was a 4-bay storage device, connecting to a single host PC/server using USB or Firewire and later eSATA. This model evolved into the Drobo S and was complemented with the 8-bay DroboPro. Finally, the DroboElite was released in November 2009, using the same form factor as the ‘Pro but with additional connectivity and features. Although the ‘Pro and ‘Elite models shared the same look and feel, they are actually different products and so there’s no upgrade process to remove drives from the ‘Pro and install into an ‘Elite shell.

Physical

The DroboElite is an 8-bay device, accommodating standard 3.5″ SATA form factor drives. The drives are inserted vertically (rather than horizontally in the 4-bay models), with drive’s top side pointing to the right. No caddy or additional support is required. From the front, there’s nothing to separate the ‘Pro and the ‘Elite as they look identical. In fact, the removable magnetic front bezel only says “Drobo” and has no other distinguishing markings. One thought worth considering for the future is providing a method of identifying multiple Drobo’s that have been installed into a rack or cabinet. The drive lights can be flashed on and off but this only lasts for a few seconds and wouldn’t help an engineer who’s alone in a data centre or machine room. This is a particularly important point as the ‘Elite is a multi-user device and so aimed at SMBs with machine rooms and rack-based equipment (a rack mount kit is available).

Drobo Elite (Bezel Removed)

Turning to the back of the ‘Elite, we start to see the differences with previous models. The major one here is the presence of two Ethernet ports, providing network redundancy and multi-pathing support for iSCSI. The power supply is still built in (and not removable), plus there’s a power switch and a USB connector – that it. I’m undecided as to whether an integrated PSU is a good or a bad thing and I flip back and forth between internal/external and multiple supplies each time I think about it. On reflection, the consideration here should be that the Drobo is not a “high availability” storage array in the modular or enterprise class. It doesn’t support mulitple controllers or other features like mirrored cache, so in reality, multiple power supplies is probably over-configuration. Obviously dual Ethernet connections is essential in a device that will support multiple hosts; this feature provides redundancy from failures in network equipment or more likely, enables maintenance to be performed on the network infrastructure without impacting the storage.

The addition of a power switch is a positive move (and essential as the power supply is now integrated). Personally I’d have preferred the power switch to be mounted behind the magnetic bezel on the front of the unit. This would mean once plumbed in, the ‘Elite (and ‘Pro for that matter) would be managed purely from the front, for disk insertions/removals.

I’ve one final comment on the physical aspects of the ‘Elite and that covers the insertion and removal of drives. I never like removing spinning active drives and unfortunately there’s still no way to either mark a drive for removal or spin it down. The unique BeyondRAID feature of the Drobo series means that any drive can be removed from a unit and the device ‘self heals’ to compensate for the loss of capacity, rebalancing blocks of data redundantly across the remaining drives. Of course drive removal doesn’t have to be done after a failure; the whole benefit of the Drobos is the capacity increase functionality (I frequently swap drives in/out as part of testing). I’d therefore like to be able to spin down or at least highlight a drive I intended to pull, via software, before I physically pull it. It gives me more confidence that I’ve pulled the correct drive when managing multiple units.

Additional specifications can be found here: http://www.drobo.com/pdf/droboelite-datasheet.pdf

In the next post, I’ll look at connectivity protocol support.

Enterprise Computing: Violin Memory Inc Release New All-SSD Array

Chris M Evans — Wed, 26 May 2010 08:22:21 +0000

I’m not a fan of making press releases on behalf of other companies however once in a while, a news item catches my interest. So it is with the announcement of the Violin Memory Inc. 3200 series of all-memory storage arrays. Why are these interesting? Because I think they are moving and potentially blurring the boundaries between spinning drives and memory-based permanent data storage.

Background

Building arrays from pure memory isn’t new; Texas Memory Systems have had the RamSan series of products on the market for some time now (and there are others out there). Of course, the problem for many large organisations is how to make use of such an expensive and relatively small device. There are plenty of use cases where flash/SSD may be useful, however (a) it is difficult to target exactly which applications and (b) for those applications that can be identified, potentially only part of the data will benefit from acceleration.

One solution has been to follow the route of the traditional vendors and add SSD as an extra device within the same hardware chassis. This isn’t a solution to using SSD but rather a sticking plaster over the problem; the SSD may give better read performance but it is unlikely that writes will be accelerated to the level justified by the additional costs of the SSD device itself. In addition, the SSD is sitting behind a traditional storage array. Vendors such as EMC, IBM and Hitachi have spent millions of man-hours and hundreds of millions of dollars on software developments to help smooth the impact and manage the unpredictable performance of hard drives. Remember that when an I/O request is received, the storage array has no idea where a mechanical device like a hard drive is positioned and so cache, algorithms and that other clever intellectual property have been used to mask these physical inadequacies.

However, despite vendors’ best efforts, spikes and unpredictable response times do occur and there’s no way to remove them and guarantee completely consistent I/O responses.

The Violin Approach

So what happens if you can remove the cost issues and buy an SSD-based array for the same price as tier 1 storage? This is the route Violin Memory are taking to market – make the SSD storage array as closely priced to tier 1 arrays as possible. Remove the thought process and complications of determining what to place on SSD by making the price argument irrelevant.

In reality, Violin haven’t reached that price parity yet; prices are quoted around the $20/GB mark, which is around double what I’d expect to see for tier 1 storage (depending on volume). However it is in the order of magnitude where organisations can look at those troublesome applications that decide that the cost of additional servers, disk spindles or re-writing the application is outweighed by simply moving the application to a Violin SSD device.

I think this is the ultimate tipping point for SSD use; where the cost of improving application performance is exceeded by the cost of moving to SSD, then SSD will win. Where improving application performance is justified by increased business advantage, the business case is written.

Tech Specs

OK, let’s have a look at the technical specifications for the techies amongst you. Firstly, today’s device capacity sits at 10TB in 3U and is expected to grow to 20TB in Q3. I’ve also been told that this capacity is expected to be close to 5x greater by the end of 2010, which means 100TB of memory-based storage in a 3U unit.

The 3200 supports PCIe (x4 & x8) as well as 4/8Gb Fibre Channel and 10Gb iSCSI and FCoE. Latency is less than 100 microseconds.

Violin array use VIMMs (Violin’s name for their flash memory cards. These are grouped together into 1TB units, using RAID-5 technology to manage failures. Maintenance can be performed online periodically to replace failed VIMM devices.

There’s one major issue with Flash/memory-based arrays that Violin claim to have addressed. That is the issue of degraded performance over time. Have a look at the following graphic, showing saturated workload on the Crucial C300 versus X25M from Intel. This graph and the associated review can be found on Anandtech’s website here. Very quickly with heavy use, the performance for these devices drops off. Violin claim their array doesn’t suffer similar issues and can deliver sustained performance. Of course, we can believe that statement once we’ve seen a review of the product delivering the performance as promised.

Futures

A 10/20TB capacity in 3U isn’t huge by today’s standards. If Violin Memory can deliver on their promises and bring a 3 to 5-fold increase in performance by year end (with a continual reduction in price) then things start to look interesting. I’d like to see the results of some long-term stress tests on the 3200 series devices. I have some more material to post in the coming days, once I can validate what’s open and not under NDA/embarbgo. In the meantime, here are some questions to ponder:

Do I have any I/O bound applications?
Can I measure/determine my I/O bound applications?
Is there direct businss advantage from increasing I/O throughput?

If you can start answering yes to the above questions, then perhaps SSD-based arrays are for you.

Enterprise Computing: Is 2010 The Year for iSCSI or FCoE? Place Your Bets!

Chris M Evans — Mon, 11 Jan 2010 22:03:53 +0000

It’s pretty traditional to start the year looking at what will be prevalent in the coming 12 months. In this opinion article, I will discuss whether we’ll see the prevalence of iSCSI or FCoE and what we can expect, looking back this time in 2011.

History and Background on iSCSI

iSCSI as a concept was first developed by IBM in 1998. With work from Cisco, a first version of the iSCSI standard was presented to the IETF in 2000. Today we see iSCSI implementations in enterprise storage arrays from the major vendors, from mid-range storage array manufacturers and in virtual appliances. The iSCSI initiator (the host component for making connections to an iSCSI device) is standard in major operating systems such as Windows and Linux. In fact, some vendors like HP/Lefthand and Dell/Equallogic make a virtue of deploying their technology as iSCSI only solutions.

However despite the ubiquitous nature of the iSCSI technology, I don’t think the adoption level is particularly prevalent in the majority of enterprise and mid-range environments, with fibre channel remaining the dominant player.

History and Background on FCoE

By comparison, Fibre Channel over Ethernet is a relatively new technology. In fact, development to ratification has only taken two years and was approved in June 2009. FCoE enables the fibre channel protocol to be passed over 10Gb Ethernet natively. This is a distinct difference from iSCSI which relies on using the TCP/IP stack for communication. As FCoE is so new, it isn’t widely adopted or supported by vendors today, with notably Cisco, EMC and Netapp being the main leaders in offering products or announcing future support.

Why iSCSI Has Failed

It’s a bold suggestion, but I personally think iSCSI has failed to live up to the hype surrounding its development. Although iSCSI is cheaper to implement and support, Fibre Channel remains the dominant force in storage networking today. Why is that? It’s a question I’ve answered many times, there are a number of reasons:

Legacy of deployed technology. Fibre Channel became entrenched in storage architecture before iSCSI arrived on the scene. Many organisations made investments in skills and hardware, which once established made FC the de facto protocol for storage networking.
Hardware interoperability. iSCSI and Fibre Channel don’t inherently work together. As far as I am aware, Cisco were the only major vendor to offer a product that had any kind of FC/iSCSI bridge functionality. This makes the two protocols pretty distinct and there’s no benefit to running multiple disparate storage protocols within the same environment.
The Cultural Issue. Fibre Channel devices have typically been managed by the storage team. iSCSI and IP are the domain of the networking team and “Never the Twain Shall Meet”. Even though iSCSI is recommended to have a dedicated infrastructure (which should still work out cheaper than FC), the hardware configuration and deployment would be managed by the network teams. There are very few instances I’ve seen (and certainly not viable ones) where Storage and Networks work in harmony.

Why FCoE will Succeed

Where iSCSI has failed, FCoE will surely succeed. As a protocol, it offers more opportunity of integration between “traditional” fibre channel and FCoE. In fact, hybrid devices are already available from the market leaders, Cisco and Brocade. FCoE is being built into the hardware designs of integrated solutions from Cisco and Acadia. Solutions from HP and IBM will have to offer the same functionality. The whole culture issue will be removed by the deployment of end-to-end architectures. Perhaps in addition the two teams may just have to learn to live with each other.

A Mixed Landscape

I think in the next 12 months we’ll see the gradual adoption of FCoE as legacy FC environments come up for renewal and the choice between 10Gb/e and 10G FC has to be made. There will no doubt be a lot of confusion as to which protocol suits large environments best. My negativity towards FCoE has been based on exactly what benefit FCoE would provide in large environments over FC. After all, cost is hardly the issue when FCoE CNAs are still so expensive. Overall, iSCSI will be the loser and will be relegated to mid-range and SOHO solutions where it does an excellent job. In 12 months time, we’ll be discussing the battle between FCoE and FC and iSCSI will be an also ran.

Virtualisation: Virtualising MY SOHO Infrastructure

Chris M Evans — Wed, 02 Dec 2009 12:48:29 +0000

I’m in the process of migrating my existing physical infrastructure into a virtual environment. There’s nothing like “eating your own dog food” (there’s got to be a better expression than that) for testing out your beliefs on how technology should be implemented, so it’s only fitting I virtualise the IT infrastructure I rely on.

In summary, I run a “production” Windows AD domain, Exchange, IIS and file services. This is a typical scenario for many small businesses, with perhaps a few variations thrown in for good measure, such as SharePoint. I’m moving to a virtual environment that is built on ESXi v4.0, a custom design Intel-based server with 2x Quad Core Intel 5420 processors and 16GB of memory. Local hard drives include 2x SAS 7.2K 500GB drives and 2x 73GB SAS 15K drives.

Storage Choices

The first question I posed for my infrastructure is to decide on a storage platform. As this is a SOHO type deployment, I don’t have the luxury of a fibre channel SAN environment. I expect that many SMBs will feel the same and not want to commit to fibre channel deployments when the alternative options out there are low cost and probably more suitable. Therefore here are my choices:

Internal drives. I have over a terabyte of internal storage but unfortunately its not RAID protected. The disks are deployed on SAS connectors on the motherboard rather than through a dedicated RAID card. Although I could change the configuration and include a card, there are a number of issues; (a) increased cost (b) downtime to replace failed drives (c) physical intervention and downtime on the server to replace drives. My goal is to deploy this infrastructure to run 24×7, so having to take the system down to replace a drive is not an option. I also think that internal drives wouldn’t give me the scalability I might need.
iSCSI. In this environment, iSCSI is a great solution; there are lots of SMB iSCSI solutions on the marketplace today, offering low cost storage with simple operations. iSCSI is definitely a possibility.
NAS. Using NAS (more specifically NFS) for storage on VMware provides lots of flexibility. The management of the filesystem itself moves to the NAS hardware and so any VMware host files appear as standard files on the NAS device. When people see this, they usually get the “light bulb moment” as they realise the possibilities this kind of deployment offers. Whole virtual machines can be cloned, backed up or even simply copied as files around the infrastructure. NAS offers easy access to log and configuration files too.

So, for me the choice is NAS. I have iSCSI deployed, but for my production environments, NAS will be used to store my VMware hosts. This is for a variety of reasons, however there’s one additional benefit I haven’t discussed and that’s portability. Placing my vmdk’s on NAS means I can move them around at will. I can replicate them, I can back them up and I can move them easily to another NAS device without using ESXi as the data mover.

Stay tuned for more discussions as I evolve my setup.

Enterprise Computing: Is iSCSI The New Home Protocol?

Chris M Evans — Thu, 26 Nov 2009 09:44:57 +0000

I’d like to think I’ve visited a lot of customer sites over the years. Admittedly most of these are “enterprise” class with multi-terabyte if not petabyte quantities of storage. None of those customers have ever bothered deploying iSCSI as their storage protocol. Invariably block storage has been implemented using fibre channel and file using CFS or NFS. Somehow iSCSI just doesn’t seem to figure. I have a few thoughts on why…

Network versus Storage. There’s no doubt, Network and Storage teams get on about as well as cats and dogs. Although both support networking technologies, they are implemented fundamentally differently. In fact the only organisation I’ve seen that had the Network team managing fibre channel had implemented it like an IP network and it was a mess. As the iSCSI protocol means handing control of the physical transport layer to the network team, then the simpler option is to avoid using iSCSI in the first place.
It’s Too Easy. This may seem like a contradiction, but running a fibre channel network usually means managing a controlled environment. Nobody connects without permission, nobody gets access without being zoned in. Implementing iSCSI is simple and so inherently means less control.
There are no Standards. Fibre channel networks are great because you have to use expensive components and match everything against approved matrices or you don’t get support. iSCSI can be implemented using the cheapest NIC and virtual iSCSI targets. However, this ease of use also means there’s no vendor certification in the way there is with fibre channel. Who are you going to blame when things go wrong?
FCoE will Rule The World. Yes, Fibre Channel over Ethernet will be the One Storage Protocol to Rule Them All and replace fibre channel, iSCSI, AoE, NFS, CIFS, and any other protocol you care to name. OK, I’m being slightly sarcastic, but FCoE is set to harmonise the physical connect, leaving iSCSI redundant.

So what’s the future for iSCSI? We’ve seen the rise in popularity of home storage devices in recent years (think Iomega and Drobo). We’re now seeing these devices sporting Ethernet connectivity that supports iSCSI. With iSCSI Initiators (like the one in Windows) being totally ubiquitous, it’s a no-brainer to deploy iSCSI in home and small office environments. Maybe there is still a future for iSCSI after all in providing low-cost block storage for the consumer masses.

Review: StarWind Enterprise Server – Part II

Chris M Evans — Wed, 28 Oct 2009 07:37:46 +0000

This is the second of a series of posts reviewing the StarWind Enterprise Server iSCSI software target. The posts relate to version 4 of the product. Previous post(s) can be found at:

Review: StarWind Enterprise Server – Part I

In the first post of this review, I explained how StarWind’s server product provided the ability to turn a Windows server into an iSCSI storage device. In this post, I’ll focus more on the specific LUN types available to the user.

Image File Device – this type is a standard iSCSI LUN – fully occupying the disk space of the creation size. LUNs can be both compressed and encrypted, however it’s not clear from the documentation exactly how these features are implemented and to what security standards.

RAM Drive Device – an iSCSI LUN emulated purely in memory on the server. This device exists purely in memory on the server and so is a volatile device; if the power goes and the server reboots, then the data is lost. I’m not sure of the merits of this kind of device when the data is so transient.

Virtual DVD Device – this device type presents an ISO file as a iSCSI DVD-ROM device. This useful for sharing out ISOs across multiple clients; up to 16 may connect at any one time. One drawback though; the ISO file must exist on the local machine running StarWind, which is sort of limiting.

Disk Bridge Device – a passthrough device representing a single physical disk on the server. Think of this in the way RDM devices work on VMware – the device itself is virtualised through to the host.

Snapshot and CDP Device – this is an iSCSI LUN which supports Point In Time snapshots and thin provisioning. A LUN can be created from an existing device; either as a snapshot or full/linked clone. The thin functionality is pretty obvious (more on this in a moment). As for the CDP functionality, I had issues attaching this device type to my Windows client, which wouldn’t let me assign a drive letter to the LUN. In addition, the volume was already formatted as a raw device.

Mirror Device – an iSCSI LUN mirrored with RAID-1 or replicated to another server. Mirroring of the logical iSCSI LUN can be done on the local machine (providing some level of fault tolerance in a single server with multiple hard drives) or using a remote LUN on another server capable of presenting out iSCSI devices.

SPTI Device – a raw (non-emulated) version of a Disk Bridge Device. This is more closely aligned to an RDM device then the Disk Bridge type. All devices (including read-only media such as DVD-ROM) can be externalised as iSCSI devices.

Virtual Tape Device – an iSCSI LUN representing a tape device. This feature has been removed from the current product and is now available separately as StarWind VTL.

There’s not doubt the range of supported devices is comprehensive, although some features such as encryption and compression need more explanation to understand how they are implemented and how data could be recovered in a server failure scenario.

Thin Provisioning

It’s pretty much taken for granted that thin provisioning needs to be provided as a feature in storage arrays, whether physical or virtual. Fortunately StarWind Server offers this as part of the CDP device type. As a test, I created an iSCSI thin LUN using the console and connected it to a Windows host. As expected, copying files to the LUN grew the device by the size of the copied files. Unfortunately, deleting the files and/or reformatting the drive did not reclaim the released space. In fact, a full format of the drive, rather than a quick format resulted in the full capacity of the drive being allocated and the benefits of thin provisioning wasted. (Tip: Use Quick format on Windows). In this release of the product there appears to be no way to reclaim unused resources within thin provisioned devices. Thin Provisioning implementations need to add this functionality as the value of using thin LUNs simply diminishes over time.

Summary

Overall, the product offers a wide range of device types. A more detailed understanding of some specifics (e.g. encryption standards) is required. Thin LUNs need some space release technology added to ensure thin doesn’t simply revert to thick over time.

In the next post, I’ll look at security, performance and the competition.

Disclaimer: I have no connection or relationship to StarWind Software. A free copy of the StarWind Server software was provided by the company for this evaluation.

Cloud Computing: Block-Based Storage

Chris M Evans — Thu, 08 Oct 2009 21:22:55 +0000

A while back, I discussed speculation from EMC around Emulex’s proposed cloud-block storage appliance, E3s (Enterprise Elastic Storage). With my current focus on Cloud Storage, I thought it would be good to delve a bit deeper into some of the aspects of why block-based cloud computing could prove tricky and why without an appliance it may be impossible.

Block Storage Legacy

Today, block-based I/O still uses the SCSI protocol to communicate between a host/server and a disk storage device. SCSI has been around since 1981 when devices were physically connected to the server itself using a controller board and old-style ribbon cables. Obviously we’ve progressed somewhat since then and seen the virtualisation of the physical SCSI interface into Fibre Channel and IP SCSI implementations (and in the future FCoE). Both FC and iSCSI have removed the need for a dedicated SCSI controller card and replaced it with Host Bus Adaptors (HBAs) and Network Interface Cards (NICs). Irrespective of this, the underlying communication protocol remains the same and the concept of the “Initiator” (host/server) and “Target” (storage device) persist today. The Initiator starts (or initiates) an I/O request; the target services that request, reading or writing data.

Writing In Blocks

We need to focus for a moment on the concept of block-based I/O versus file-based I/O. Block-based I/O has no concept of the format of the data being written to the block device (let’s call it a LUN). This is in contrast to file-based I/O where the storage device understands the data format and manages the content accordingly, ensuring data access is serialised correctly and that files are held in a logical structure (a file system). Unfortunately block-based I/O is just “dumb” storage and the host itself is responsible for overlaying a file system onto block-based devices. These JBBDs (“Just a Block-Based Device”) can be used singly or combined in complex ways to create the file system the host sees. This combination can be achieved using native Logical Volume Managers (LVMs) on the operating system, or add-ons such as Veritas Volume Manager. Consequently, an individual LUN could contain an entire file system or only a small component of one. Either way, the host operating system depends on one feature of the storage device to ensure data integrity and that’s Write Order Consistency.

Write Order Consistency

Preserving the order in which data is written to disk is a fundamental requirement for modern Journalling file systems like NTFS. Retaining write consistency ensures the file system can be recovered in the event of a server failure or failure of the link between server and disk. Ordered writes are also essential where data is replicated from one storage array to another usually at a remote location. In the event that the primary array is lost, the file system can be recovered in a consistent fashion on the remote device, even if it isn’t 100% up to date.

Remote replication can occur either synchronously or asynchronously. In synchronous mode, I/Os are acknowledged from the remote array before the I/O is confirmed as completed to the host. This ensures both the primary and remote copies are write-order consistent because the host doesn’t receive acknowledgement of I/O completion until both the local and remote copies are written. Write-order consistency is implicitly guaranteed. However, the penalty for this guarantee is the increase in latency (or host I/O response time) which results and increases with the distance between the two devices. Asynchronous replication is slightly different. Write I/Os to the primary array are acknowledged immediately, then queued up for writing to the remote device. The delay in writing data to the remote location is dependent on the bandwidth available and the latency (effectively the distance) between the devices. In any event, as long as write-order is maintained, the remote copy can be recovered even if it doesn’t represent the absolute latest copy of data. One last consideration. We touched earlier on how LVMs can combine single LUNs to create complex file systems. When asychronous replication is involved, write I/O for a single entity like a file system need to be treated together for write-order consistency. Therefore Consistency Groups allow multiple LUNs to be grouped together for write ordering, ensuring that all I/O for the file system is ordered correctly. This requirement doesn’t exist for synchronous replication as the consistency is guaranteed by ensuring the write I/O has completed on both the source and target array before acknowledgement to the host.

The Need for an Appliance

Knowing how write-ordering affects consistency is important in understanding how a block-based device would be replicated into “the cloud”. Due to latency issues, it is unlikely that synchronous replication would be offered as a method of replicating data from a host server into Amazon S3 or EMC Atmos. Instead, the most likely process will be asynchronous processing and that means installation of a dedicated appliance. The question is, where in the data path should it sit?

The Splitter

There’s no requirement for a host/server to be connected into a storage array in order to utilise cloud storage. Instead, at some point in the data path between host and local disk, a copy of the write I/O needs to be taken. This could occur at the file system level using a host agent or in a SAN environment could happen in the fabric or from the array itself. Wherever data is taken from, some form of “I/O splitter” is needed to capture write I/O as it is being transferred to disk. This technology already exists today in products like EMC’s RecoverPoint and Brocade’s Data Mobility Manager.

So here are our requirements for a block-based storage protocol:

Write Order Consistency
Consistency groups
An I/O splitter or replicator

A Theoretical Implementation

Here’s how I think block-based storage could be implemented. I’ll use the Atmos and Amazon S3 protocols to demonstrate the process. Firstly, data will be stored in blocks. Both S3 and Atmos store data as objects and so each object will need to represent a block. The file system structure can be used to store individual LUNs, with a directory representing a LUN. For example LUN 12, block 12343 could have the object name:

S3Array1LUN1212343

It’s worth noting that there’s a distinct difference between the way Atmos and S3 implement updates to objects. S3 replaces the entire object, whereas Atmos allows part of an object to be updated. So, Atmos could store an entire LUN as an object whereas S3 can’t, unless the entire LUN is replaced on each write. Clearly that’s impractical but does indicate that each API implementation will have certain benefits and disadvantages.

So, how big are these blocks going to be? Ideally, they’d be as small as a typical hard drive block at 512 bytes, however blocks of this size will seriously hamper throughput if write consistency is to be maintained; imagine 5ms latency into the cloud; that’s only 200 IOPS and consequently a throughput of 100KB/s. What’s more practical is a block size matching the O/S file system and/or database, say 8KB. Even at this size, with 5ms latency and a single thread of I/O, that’s only 1.6MB/s throughput.

Obviously this level of throughput is not going to be acceptable and there’s a real sticking point here. The cloud isn’t intelligent. It will write data as it’s received. There’s no locking control and the delivery mechanism could be unpredictable. If writes are issued in parallel, there’s no way to guarantee the I/Os are written to the cloud in the right order. So perhaps a different approach is required. Data writes to the target LUN need to be written in a log format, with the name of the object comprising both the block number and a sequence number. This could be something simple as follows:

AtmosArray1LUN12SequenceNumber-BlockNumber

e.g. AtmosArray1LUN12123445343434366-12343

As the LUN is written to, the sequence number (unique to the LUN or consistency group) is incremented for each write. The I/Os can then be written in parallel as the sequence numbers track what has and has not been received. At this point there are two choices – retain all the block updates (unlikely due to the growth in storage usage) or post process the writes, deleting all the written blocks where another later copy exists and where there are no gaps in the sequence numbers. If there is a gap, then the LUN writes are only guaranteed back to the point where the sequence number gap occurred. Restoring the LUN for access means processing the LUN block data before it can be read again by the host.

Summary

OK, so this post presents an idea of some of the issues involved in writing block-level data into the cloud. Data needs to retain integrity and consistency, but performance and throughput are an issue. Cloud storage has no intelligence, so writing and managing data needs to be handled somewhere, probably using an appliance. The appliance guarantees the data integrity which can’t be achieved with the cloud alone. Each Cloud Storage API implementation will have similar features, so using generic CRUD (Create/Read/Update/Delete) commands on objects representing blocks means any service could be used to store data. It also enables data to be replicated between services so vendor lock-in can be avoided.

I’d be interested in receiving feedback to see if anyone else has thought how block-based cloud storage could be achieved.

Personal Computing: DroboPro – Drobo Grows Up

Chris M Evans — Tue, 07 Apr 2009 22:15:07 +0000

There’s a new Drobo out!

The Drobo range has been expanded with the release of the DroboPro, an 8-drive big brother to the Drobo “classic”. The new model is being touted as a “business class storage array” and looks very interesting for the basic price tag of $1299 without drives.

Specifications

So, the DroboPro has capacity for up to eight 3.5″ SATA drives. With spinning rust boxes now at 2TB a drive, that’s up to 16TB of raw capacity. Clearly that amount of storage is probably way too much for a single PC and that’s where the “business” part comes in. The DroboPro supports iSCSI (through a single gigabit connection), raising the stakes by moving it into the multiple server league. It can also be rack mounted, although that does look a little weird.

There are limitations however as the DroboPro supports a maximum of 16x 16TB drives – but they are thin provisioned.

Wish List

My comments are only based on what I’ve read on the Drobo website, however I am an existing Drobo owner. I have both the Drobo “classic” and the DroboShelf (oops, sorry DroboShare). The classic is great, just running and running, however the DroboShare was a disappointment and I don’t use it for anything other than a rest for the Drobo. I’d like to evaluate the ‘Pro, however in the meantime, here’s my wish list of features:

DroboMini – please do a Drobo which accepts 2.5″ drives – preferably both SATA and SAS.
Partitioning – with multiple drive type support, provide multiple pools – tiering in a box.
Multiple Ethernet – more than one Ethernet port would be good!
Snapshots – point in time copies of LUNs
Replication – OK perhaps I’m going a bit far…

I think it’s worth recapping what the DroboPro offers; secure scalable storage – iSCSI out of the box – thin provisioning. Bring it on!