A report published today by OFGEM, the UK’s energy regulator makes bleak reading for the future of electricity supply in this country.  The BBC discusses it here.

The report highlights £200bn of under-investment and the risk of power cuts over the next decade.  It’s a damning statement on the deregulation of the power industry but should be ringing alarm bells for anyone concerned with managing their data centre.  The affects could be:

• Price Rises – consumer prices could double, therefore commercial rates will be affected too.
• Backouts – there may be periods of electricity blackouts if supplies aren’t available.
• Nationalisation – the government may have to take back control of the industry.

For the UK at least, electricity consumption in data centres is going to become more critical and the efficiencies of using virtualisation even more important.  In addition, this potentially poises Cloud Computing to gain more acceptance as workload is shifted between resliency domains in different power supply juristictions.  This doesn’t necessarily mean “public” clouds but also private clouds within organisations large enough to have multiple computing centres.

The key part will be getting the application developers for those companies to change their approach and think how applications and data can be managed in a truly distributed environment.  That’s when clouds will have their day…

My previous post regarding HDS and 3Par’s savings guarantees generated a few replies and rather than respond in comment, I’ve written an additional post in case I rant on too much.

First thought: vendor guarantees need to be genuine; no hiding behind obtuse configurations that help to fluff the figures, or restrictions on the type of data suitable for reduction.  Admittedly, this means the data to be migrated can’t be on *any* source platform but common sense dictates that if someone like 3Par are selling on the basis of thin provisioning, then the saving is going to come from using that feature.  Therefore migrations where TP is already in place probably won’t get the benefit or the guarantee.  The upshot is, make the guarantee on the basis of a feature (or features) you think the customer could implement to save money.

Second thought: why wait for a vendor to offer a guarantee?  Here’s the logic.  If you’re spending anything from £50,000 – £1,000,000 upwards, why shouldn’t you expect the product to deliver to your requirements?  Imagine you were embarking on a building project; would you give your builder a set budget and be happy if he delivered your house 20% smaller because he undersized the cost of the job?  I doubt it.

So, if you’re looking to buy a new storage array, get the vendor to contractually commit to levels of performance and availability (that remain consistent as capacity scales) with penalties if their proposed solution isn’t fit for purpose.  If they are confident in their solution, they will happily (well probably begrudgingly) sign.  If they’re not confident, then they will have a vested interest in getting the solution right in the first place, not selling it at the lowest price.

Remember, there are half a dozen or so enterprise-class vendors in the market place today.  With a well architected solution, you can use just about any of them, so you have no reason not to ensure the deal works for you….

Oh, if you’re wondering who the 6+ vendors are (in no particular order), how about these for starters: EMC, IBM, HDS, HP, Netapp, 3Par….

In the past week, both 3Par and HDS have announced a 50% guarantee on reclaimed storage if customers move data from existing “fat” legacy arrays to 3Par storage or use HDS thin provisioning technologies.  The 3Par news release is here; the Hitachi news release is here.

The 3Par guarantee is presented with a caveat:

The above is intended to highlight certain aspects of our Get Thin Guarantee and does not contain the full terms, conditions, limitations, definitions, and other provisions (“Terms”) of the Get Thin Guarantee. The Terms shall be contained in a written Get Thin Offer which shall take precedence over the above. Qualification for our Get Thin Guarantee is subject to your acceptance of a Get Thin Offer containing the Terms and satisfaction of those Terms.

This, no doubt is the legalese to satisfy the lawers, however it probably also holds the specific terms and conditions of the data to which the 50% applies.

Hitachi’s caveat restricts the 50% savings to migrations from RAID-1 to dynamic provisioned RAID-5, otherwise the guarantee is only 20%.  Well duh.  Moving from RAID-1 to RAID-5 alone gives a saving of up to 42% depending on how you choose to calculate it.

You may remember Netapp also have a 50% space guarantee (details here).  I wonder how that is going….

It too had a number of restrictions on the type of data which would be classed as suitable and also required customers to implement the storage in a certain way.

Whilst these sorts of programs are welcome if they genuinely result in savings for customers, what we need to be wary of is clever marketing which in reality results in configurations that customers wouldn’t implement.

One final thought; Hitachi guarantee to cough up the difference between used and predicted savings in storage.  That’s great (and I don’t know if Netapp and 3Par agreed the same), but there’s always a tradeoff between storage efficiency and performance.  Will the vendors also guarantee that performance won’t be impacted?

One final final thought…. I bet EMC don’t try and match the competition…

They say that you learn the most when you make mistakes and things go wrong.  Well, last night I certainly must have learned a lot.  What started as a simple physical re-organisation of my hardware turned into a rebuild of my production VMware ESXi server – finishing at 1am.  Here’s what happened.

Failing Disk

I started by shutting down and moving my production ESXi Server out and back into the standard rack it occupies.  On power up, the server failed to reboot, claiming the boot disk was no longer present.  A quick check inside showed that the SAS connector on the boot disk had come loose, so I plugged it back in and tried again (Oh, SAS specification guys – bad design, no retainers on the plugs).  Unfortunately, the boot disk had somehow become corrupted and the server wouldn’t come up.  No problem, I thought, just repair using the installation media.  This is where things started to get complicated.

My ESXi server runs off a Seagate Savvio 2.5″ 15K 73GB drive, one of four Seagate generously loaned me last year for long term testing.  More on that another day.  The server has two disks installed, one of which has VMs on it.  During the repair process I wasn’t sure which disk was the O/S and which was data.  ESXi doesn’t help much, only indicating that both disks contained data in partitions, data that would be lost if I reinstalled. 

Lesson 1 – Make sure you know exactly how your hardware is configured, down to the SAS ports each drive is plugged into.

Actually having multiple drives of the same type is a pain.  So rather than risk data loss, I removed both drives and re-installed the ESXi O/S from a third Savvio drive.  All good.  Now I need to locate and import all my VMs, however some were on the removed Savvio disks.  This meant installing each disk independently and checking the contents to determine which contained VMs and which contained the broken O/S.

Lesson 2 – Wherever possible, place your VMs on disks separate from the server itself.

Yes, I do have most of my VMs on my Iomega ix4-200d, but, rather crucially, not my Windows 2008 AD Server, which needed to be moved from internal disk to the ix4 before I continued (schoolboy error there).  The AD server was rather important for accessing my, ahem, ix4, which is configured to validate logins using AD.  This creates a bit of a circular reference which could have been a disaster.

Lesson 3 – Place your Windows domain controller on a physical server, or have another independent backup elsewhere.

Having a physical server just for AD control isn’t part of my total virtualisation plan, so I’m looking at whether I can host a backup controller with Amazon AWS and use VPN to secure it into my private network.  This way, if I ever have an issue, I can still authenticate.  The issue of course is cost, which may make a dedicated server the cheaper option.

So, by 1am everything was back up and running.  Did I learn anything else?  Well yes…

Lesson 4 – after 22 years in IT, I should remember that adequate documentation and a DR plan are crucial.  In fact, in a virtualised environment, they are essential due to the concentration of risk placing all systems on a single server causes.

So what next for my virtual infrastructure?  I have a few changes planned; I’ll create a backup ESXi server that can import and run the VMs in the event of a future server failure.  I will also be investigating AWS with Windows 2008 and VPN to create a backup domain controller and see if I can continue to work if both server’s hardware failed.

That leaves one Single Point of Failure… my ix4-200d.  Anyone want to donate me a spare one?

It’s pretty traditional to start the year looking at what will be prevalent in the coming 12 months.  In this opinion article, I will discuss whether we’ll see the prevalence of iSCSI or FCoE and what we can expect, looking back this time in 2011.

History and Background on iSCSI

iSCSI as a concept was first developed by IBM in 1998.  With work from Cisco, a first version of the iSCSI standard was presented to the IETF in 2000.  Today we see iSCSI implementations in enterprise storage arrays from the major vendors, from mid-range storage array manufacturers and in virtual appliances.  The iSCSI initiator (the host component for making connections to an iSCSI device) is standard in major operating systems such as Windows and Linux.  In fact, some vendors like HP/Lefthand and Dell/Equallogic make a virtue of deploying their technology as iSCSI only solutions.

However despite the ubiquitous nature of the iSCSI technology, I don’t think the adoption level is particularly prevalent in the majority of enterprise and mid-range environments, with fibre channel remaining the dominant player.

History and Background on FCoE

By comparison, Fibre Channel over Ethernet is a relatively new technology.  In fact, development to ratification has only taken two years and was approved in June 2009.  FCoE enables the fibre channel protocol to be passed over 10Gb Ethernet natively.  This is a distinct difference from iSCSI which relies on using the TCP/IP stack for communication.  As FCoE is so new, it isn’t widely adopted or supported by vendors today, with notably Cisco, EMC and Netapp being the main leaders in offering products or announcing future support.

Why iSCSI Has Failed

It’s a bold suggestion, but I personally think iSCSI has failed to live up to the hype surrounding its development.  Although iSCSI is cheaper to implement and support, Fibre Channel remains the dominant force in storage networking today.  Why is that?  It’s a question I’ve answered many times, there are a number of reasons:

• Legacy of deployed technology. Fibre Channel became entrenched in storage architecture before iSCSI arrived on the scene.  Many organisations made investments in skills and hardware, which once established made FC the de facto protocol for storage networking.
• Hardware interoperability. iSCSI and Fibre Channel don’t inherently work together.  As far as I am aware, Cisco were the only major vendor to offer a product that had any kind of FC/iSCSI bridge functionality.  This makes the two protocols pretty distinct and there’s no benefit to running multiple disparate storage protocols within the same environment.
• The Cultural Issue. Fibre Channel devices have typically been managed by the storage team.  iSCSI and IP are the domain of the networking team and “Never the Twain Shall Meet”.  Even though iSCSI is recommended to have a dedicated infrastructure (which should still work out cheaper than FC), the hardware configuration and deployment would be managed by the network teams.  There are very few instances I’ve seen (and certainly not viable ones) where Storage and Networks work in harmony.

Why FCoE will Succeed

Where iSCSI has failed, FCoE will surely succeed.  As a protocol, it offers more opportunity of integration between “traditional” fibre channel and FCoE.  In fact, hybrid devices are already available from the market leaders, Cisco and Brocade.  FCoE is being built into the hardware designs of integrated solutions from Cisco and Acadia.  Solutions from HP and IBM will have to offer the same functionality.  The whole culture issue will be removed by the deployment of end-to-end architectures.  Perhaps in addition the two teams may just have to learn to live with each other.

A Mixed Landscape

I think in the next 12 months we’ll see the gradual adoption of FCoE as legacy FC environments come up for renewal and the choice between 10Gb/e and 10G FC has to be made.  There will no doubt be a lot of confusion as to which protocol suits large environments best.  My negativity towards FCoE has been based on exactly what benefit FCoE would provide in large environments over FC.  After all, cost is hardly the issue when FCoE CNAs are still so expensive.  Overall, iSCSI will be the loser and will be relegated to mid-range and SOHO solutions where it does an excellent job.  In 12 months time, we’ll be discussing the battle between FCoE and FC and iSCSI will be an also ran.

This post is a guest posting from Jerry Huang from Gladinet and discusses the differences between Microsoft’s cloud platform Azure and the Amazon S3 platform. 

 This month, Microsoft transitioned Windows Azure Platform from public preview mode to full production mode. Azure Storage is part of the Azure Platform and competes directly with Amazon S3. It is great news for cloud storage consumers but which one to choose?

This article will compare the price, performance, developer support and partner support between the two.

Round 1: Price

Microsoft has great introductory offer to celebrate the commercial launch of Azure. As shown below, everyone can get 500MB free storage and everything in January 2010 is free (check price here : http://www.microsoft.com/WindowsAzure/offers/?lang=en&country=US&offer=MS-AZR-0001P ). There are additional benefits for MSDN subscribers such as 10G free for 8 months with additional ongoing benefits (check MSDN benefits here: http://msdn.microsoft.com/en-gb/subscriptions/ee461076.aspx).

As a counter measure from Amazon S3, it offers free transfer until June 30, 2010 (check price: http://aws.amazon.com/s3/#pricing ).

Other than these free offers, the price between Azure Storage and Amazon S3 is pretty much the same with $0.150 per GB. Since Amazon is more established, it favors big customers since the price will be lower if you have more than 50TB of data. 

In round 1, Microsoft has the edge on price with free introductory offers, especially for MSDN subscribers.

Score: Microsoft Azure Storage 1  | Amazon S3 0

Round 2: Performance

For performance, we will need to compare the upload and download speed from the same location to both Azure Storage and Amazon S3. We will use a tool – Gladinet Cloud Desktop (http://www.gladinet.com ), which can map both Azure Blob Storage and Amazon S3 side by side as virtual folders in a network drive.

The test will be simple, drag a big file from local PC to Azure first for the upload, then drag it from Azure to another place on local PC for the download. Then repeat the same steps for Amazon S3. We will compare the time it takes afterwards.

File Size : 15,903,611 bytes

In round  2, Azure has better upload time while Amazon S3 has better download time.  Considering Amazon S3 may have more usage, having a shorter download time has an edge because it means  a closer data center.  However, if your use case is online backup – write once and seldom read, you may favor upload speed.

The test result may also vary from location to location so one person is close to Azure while the other person is close to Amazon data center and vice versa.  I would recommend using Gladinet Cloud Desktop to test the water for both before committing  to either one.  For important documents, you may want to consider using both.

At the end, both are fast enough for daily usage.

Score: Microsoft Azure Storage 1  | Amazon S3 1

Round 3: Tools

As it is now, doing simple Google Search on “Amazon S3″ and “Azure Blob Storage” reveals more tools for the Amazon S3 than for Azure Blob Storage.  Amazon S3 holds the edge now. However,  it is unthinkable that Microsoft will not catch up with all its software and platform power, given it time. With Azure integrated into Visual Studio,  I would expect more tools to show up for Azure Storage.

Score: Microsoft Azure Storage 0   | Amazon S3 1

Round 4: Partners

Amazon S3 has been powering several well known online storage companies, such as Dropbox, JungleDisk and etc. These vendors have locked into Amazon S3 and unlikely to switch. Microsoft Azure has its own same-company partners in Windows Live (such as Live Mesh & SkyDrive) too.

Score: Microsoft Azure Storage 1  | Amazon S3 1

Conclusion

Windows Azure Storage is an exciting cloud storage offering, competing with Amzon S3 directly.  The competition is healthy with consumers have more choices and more free options.

Reference

1. Map Drive and Backup to Azure Storage in 3 Steps (http://gladinet.blogspot.com/2010/01/map-drive-and-backup-to-windows-azure.html )
2. Manage Azure Blob Storage with Ease!  (http://gladinet.blogspot.com/2009/12/manage-azure-blob-storage-with-ease.html )

In my first review of the DroboPro, I was somewhat disappointed that I couldn’t access the ‘Pro from the Drobo Dashboard when the device is serving out iSCSI LUNs across the IP connection.  This restriction is a serious flaw, as it requires taking the DroboPro down and off the network in order to connect to a PC running the Drobo Dashboard.  However, I’ve found a workaround that allows me to run the Dashboard *and* present iSCSI LUNs at the same time.  Here’s how…

Firstly, let’s review my configuration.  I have a DroboPro that connects to my main ESXi server.  It presents eight 2TB thin provisioned iSCSI LUNs to ESXi.  These are then used as datastores or can be used for RDM devices.  It’s the use of RDM devices that proves the answer to the problem.  An RDM (Raw Device Mapping) device isn’t emulated by vSphere/ESXi as a standard vmdk would be.  ESXi passes all I/O directly to the LUN, simply retaining a pointer reference to it on an existing datastore.  I’ve used RDMs in the past to virtualise storage management servers (for example EMC’s Solutions Enabler) where the software talks directly to a LUN on the storage array using bespoke non-standard SCSI commands.  It appears that the Drobo dashboard also uses bespoke commands to connect to the DroboPro when only the iSCSI IP link is available.  Fortunately, RDM honours these too, so the DroboPro can be administered through the iSCSI LUN using it as a command device.

So, to administer your DroboPro:

1. Create a dedicated iSCSI LUN on the DroboPro using standard techniques.  The LUN can be created as a minimum of 1TB, however as its virtual, the LUN will occupy only a tiny amount of real physical space.
2. Assign the iSCSI LUN as an RDM device through ESX/ESXi to a Windows host.  Format the LUN on the host.
3. Install Drobo Dashboard on the same Windows host.  Voila!

It may well not be necessary to create a dedicated LUN, as I tested this theory using an existing server that had a DroboPro iSCSI LUN already assigned to it.  There’s one thing to be careful of though.  Remember if the virtual server you’re running Drobo Dashboard on is configured from LUNs on the DroboPro, you need to ensure you don’t affect those LUNs otherwise you compromise your management station.

Finding this workaround makes me feel happier about using the DroboPro in an ESXi/ESX environment.  I’d still recommend creating all the volumes you need on the DroboPro up-front before you start using it with ESX, however at least now there’s a way to monitor the device without relying only on the external lights.

This is a problem that’s been on my mind for some time; can the Drobo Dashboard see more than one device at the same time?  Well, it turns out the answer is yes.

At first my thought was that it wasn’t possible for the Drobo Dashboard to manage more than one device.  Currently I have a Drobo connected into a DroboShare and a shiny new DroboPro, which is serving iSCSI LUNs to my ESX server.  I’ve not been able to get Drobo Dashboard on my main Windows 7 PC to work correctly.  It seems to have issues seeing the DroboShare.  I’ve checked all the obvious stuff (like firewalls) but still can’t get it to work.  Anyway, I was testing connectivity from my MacBook to the DroboPro using USB and to my surprise, both the DroboShare (called Drobo1) and my DroboPro (called DroboNAS2) both appeared in the GUI.  Have a look at the attached graphic.  You can see the Drobos listed at the top of the screen; this can be used to toggle between a display of both devices.

Now this is the Mac version of the Dashboard.  I can’t vouch for the Windows version as mine clearly doesn’t work correctly, however I imagine it works the same way.  So the answer to the question; “Can Drobo Dashboard manage Multiple Drobos?” is clearly – Yes.

Oh, one other thing worthy of note; I can confirm that the DroboPro definitely will *not* let me connect to the USB port while it is serving iSCSI LUNs through IP.  That’s a real shame, as I can’t manage the device without an outage.  Bit of a showstopper, I think.

Thanks to everyone who’s taken the time to read my semi-focused rants over the past 12 months.  I’ve been grateful for the interaction and comments, good or bad.

You’ll see the site has a new design.  Over the coming 12 months I hope to blog more than ever and to cover technologies in more depth and breadth. 

Wherever you are in the world, thanks for your contribution and a happy new year to you all!  Let’s look forward to a tech 2010!

This is the time of year I like to do a little early spring cleaning and one group of storage devices due a clean out are my old hard drives.  After many faithful years, I’m finally saying goodbye to all of my parallel ATA devices. 

One the one hand, I hate throwing things out.  I like to use technology to the absolute limit, but eventually some things outlive their usefulness and so it has come to pass for PATA hard disks.  Unfortunately the maths and environmentals no longer stack up.

Capacity and Power

I’m disposing of twelve (12) 3.5″ drives that have a total capacity of only 344GB.  That’s a mere 28GB per HDD.  The lowest capacity drive is only 1280MB in size.  Contrast that to the latest hard drive I’ve just purchased.  It’s (another) 2TB model – the Seagate Barracuda ST32000542AS.  This drive offers five (5) times more capacity for only 1/12 of the physical space and power requirements.  With these kind of gains in efficiency, why retain old drives any further? 

RAID and Speed

Now of course, multiple drives have one or two benefits over a single hard drive; firstly they can be protected using RAID and second, multiple spindles provide improved throughput.  However the cost of 1TB and 2TB drives is now so low, they can be purchased in multiples reasonably cheaply and the RAID benefits retained.

Recycling

As a final thought, I think we should be doing more to recycle old hard drives.  After all, we recycle mundane components such as printer ink cartridges, batteries and so on.  Perhaps we could see a slot for hard disks at the local recycling centre?