The Legacy View

Legacy Provisioning Proces

For the last 10 years or so, the traditional view of  storage management has consisted of a number of Storage Administrators using a GUI, CLIs and/or scripts to process storage requests as they are generated by the business user.  The process is highly manual, with lots of interactions between the requestor, the storage admin delivering the work and other intermediaries to cover things like billing, change control, capacity management and workload scheduling.  This made the overall process pretty people intensive and not surprisingly elongated the delivery time.  Many end users will also have recollections of asking for their specific requirement to be told they can only have something “off the shelf” – i.e. storage to a standard LUN size and with a specific RAID protection.  This was done for obvious reasons; firstly the configuration of large arrays was predicated on pre-planning and a fixed design, usually created at hardware installation time.  Once defined and in use, it couldn’t be changed (or at least couldn’t be changed without significant impact and cost).  Second, it makes sense to reduce requirements into a smaller subset to make the provisioning process easier.  As well as being rigid in configuration, many legacy arrays assume the creation and provisioning of LUNs is an infrequent task.  Many require requests to be packaged and executed in batch and certainly can’t cope easily with concurrent requests.  Although it is possible to automate some provisioning processes using CLIs and scripts, this doesn’t address the real requirements in creating an on-demand model.

The New World

API Provisioning Process

As we scale up to ever large IT deployments and especially within service-based or “cloud” configurations, the idea of having large amounts of human intervention in the provisioning process simply doesn’t work.  Instead, we need to move to a model of “storage on demand” where an external agent – user or orchestration software – can request storage as part of a portal and see the request actioned in real-time or at least within a matter of minutes or hours.  This kind of operation can only be delivered where the hardware has been designed for the purpose.  Where previously storage administrators were involved in every provisioning request, those requests will be actioned within a provisioning framework, defined by the administrator or a storage architect.

Framework

What do we mean by framework?  Well, it’s all about setting a set of parameters around which allocations take place.  This could include:

• LUN Size
• Resiliency/Availability
• Performance
• Security credentials
• Snapshot policy
• Capacity on demand LUN

Designed for API

Of course there’s a big question around whether APIs can be retro-fitted to existing storage.  In classic IT tradition the answer is “it depends”.  Without a doubt no new storage array should be released without native API functionality.  However fitting an API to existing technology will depend on how flexible the existing configuration process is.  It’s possible to create an API wrapper and build automation into a middleware layer.  This is how products such as iWave’s Storage Automator work.  However adding these features to existing storage products could be costly and still be an imperfect solution.

The Storage Architect Take

As new storage platforms evolve, native API support is a must.  The Storage Administrator will simply be required to deploy the infrastructure and plug it into a higher framework from where provisioning will be entirely automated.  Vendors offering this level of functionality will be the most attractive to service providers, looking to make the cost of acquiring and managing storage as cheap as possible.  We’re about to see a paradigm shift in the way in which storage is managed and possibly an end to the storage administrator.

Click here to view the embedded video.

What is It?

What is Zimbra?  If you’re not familiar with the product, it’s effectively an email server with additional functionality, providing the sort of features you’d expect with Exchange or other similar messaging products.  Zimbra is delivered on Linux (and in the appliance specifically on Ubuntu), which may cause problems for certain customers familiar with the Microsoft route for managing email services.  The packing of any product as an appliance should theoretically take out some of the knowledge needed to deploy in a virtual environment, but can appliances really make application delivery that simple?

Installation

I filled in the online form and obtained a link to download the trial software, which can be delivered in OVF/OVA format or via a URL.  The download isn’t small at around 2GB and expands to around 18GB when installed.  I installed Zimbra in my vSphere 4.0 environment as you can see by following the screenshots in the gallery at the end of this post.  On initial power up the appliance asks the standard questions on licence agreements, data storage location and then performs a first time installation.  A configuration user password is then required and at this point the remainder of the configuration is continued via a web browser.  You can see from screenshot 6 that the browser URL is provided.  The installation defaults to a DHCP IP address, so this is the opportunity to both set a fixed IP and the host name.  After a couple of failed installation attempts I found it best to get the network settings and most importantly the host name set correctly here.  Host names need to be FQDN, otherwise the antispam and antivirus engines don’t work correctly.  Although I have a working knowledge of Linux and Ubuntu, I couldn’t manage to correct a wrongly coded host name after installation.  It was easier to perform a full appliance re-install than mess about correcting the problem.

Appliance?  Good or Bad?

There’s no question that an appliance is a simple way to perform an application installation.  Having everything pre-packaged and simply answering questions makes life much easier for that first install; I saw that with Zimbra as I re-installed a number of times.  However I see appliances as having only this one-off  benefit.  They are not an answer to not understanding the underlying operating system or application.  Ask yourself the following questions:

• How will I perform maintenance on the appliance?  You can guarantee that any appliance will immediately be out of date within weeks of creation.  Patches and updates will still need to be applied over time.
• How will I upgrade my appliance?  What happens if you want to move to a later version of the operating system or application?  Specific knowledge is still required.
• How will I backup my appliance/application?  Again, platform and application knowledge is required.

Of course one solution is to continually move to new, later versions of the same application appliance.  This requires knowledge of how the data is stored within the application and development of processes to migrate the data, which may be more complex than simply understanding and managing the underlying O/S.

Zimbra Evaluation

Once I’d sorted out the teething problems with host names, the Zimbra appliance was remarkably easy to deploy.  I now have it managing mail for my “thevirtualisationarchitect.com” domain.  The administration console is web-based and written Ajax and can be seen in screen shots 8 & 9.  Most features are pretty intuitive and there’s little need to revert to the CLI to perform administrative functions.  The user GUI is also web-based and can be seen in screen shot 10.  It is very similar to Outlook Web Access in look and feel.  As Zimbra supports IMAP, configuration to Entourage was a 30-second task, removing the need to use the web GUI at all.

I’ll continue to evaluate the software for the licence period after which I have a decision to make.  All of www.thevirtualisationarchitect.com sits in the cloud – the website itself is hosted on Amazon AWS, so it seems appropriate to keep web services virtual too.

My Cloud View

Cloud is not virtualisation.  Virtualisation is virtualisation.  It’s a way of consolidating the physical entities we know today and making them available in the same manner in a virtualised form.  Moving 1000 Windows servers to 1000 virtual machines changes nothing.  You’re still offering IT to customers in the same way – using the concept of that physical entity called a server.  This is a step forward in reducing cost, but not in delivering cloud.

Cloud today is a hybrid of two things; virtualisation & services and service abstraction.  In the first form, Amazon, Rackspace and the others are offering the same physical objects (i.e. servers) but packaged nicely so they can charge you efficiently for them.  What makes this kind of environment a step above virtualisation is that you have no view, concern or interest in how, where or by whom those servers are being provided as long as they meet your requirements for security and availability.  The hosting company is effectively offering you elasticity over and above the basic virtualisation offering you could achieve yourself.  If you need more compute, then they guarantee to provide it and they manage the pain of ensuring resources are available.  Second, is service abstraction; some things are non-server dependent.  For instance; generic databases or email outsourcing.  The hosting company offers you a service, rather than a virtual device.  This leads on to the next step; what Cloud will be in the future.

To see what Cloud Computing can offer, we need to move forward away from the fixed view of infrastructure and start thinking about data and applications.  In the future I want to move away from thinking about fixed data types like files and block devices and think about objects and metadata.  These objects need to be modifiable in a way that gives me consistency, so if I lose a node, or device that’s processing the data, I can recover it from elsewhere and continue working.  I want to process my data in parallel and without geographic constraints, so I can run or move my workload anywhere; presumably to the provider who is offering me best price on that day.

The treatment of data in this form means I need to rethink how applications are written.  I can’t rely on a simple client-server hierarchical architecture but need to move to one where my processing and data is distributed.

Re-writing distributed applications will be the hardest part of delivering Cloud Computing.

Application development in a completely distributed environment creates significant challenges. Here are a few of the most obvious issues;

• How do I ensure consistency (multiple updates in multiple locations)?
• How can I manage performance?
• How can I move my data around easily and quickly?
• How can I back my data up?
• How can I move my application and/or data to make best use of cost savings?
• How can I ensure my data remains secure?

The ultimate Cloud Computing model needs to remove almost all the aspects of infrastructure structure we know today (server types, storage arrays, block devices, NAS) from the customer.  It will offer abstracted programming concepts – store, retrieve, search, update – to process data.  Customers will be charged for their consumption of these resources.  Abstraction will allow applications to be run on (almost) any environment.  These services will still be delivered on infrastructure, but we won’t care or need to care what it looks like.  Now there’s a challenge…

Background

Cloud Computing and Cloud Storage offerings are all about letting someone else deal with the headaches of IT infrastructure management.  By outsourcing to the ”cloud”, organisations may be able to significantly reduce Capital Expenditure and  the Operational costs associated with providing information technology.  Nirvanix have been offering cloud-based storage for some time, mainly through third party channels (companies like Atempo and FreeDrive), but CloudNAS is a direct offering, integrating with a standard Nirvanix account.

The premise of the software is simple; use a standard Windows server as a gateway to virtually unlimited storage in the cloud.  A Windows server (virtual or physical) acts as a standard Windows File Server and therefore storage can be divided out through multiple shares and be integrated into the Active Directory security model.

Testing Testing

I installed the CloudNAS software on a Windows 2008 Server host on my VMware testing farm, meeting the minimum requirements of 2GB of memory and 2Ghz processor.  The installation process requires SDN account details; userid/password and the details of a specific application under SDN created for CloudNAS.  If you’re not familiar with the way SDN works, a user account can have specific applications against which data is stored.  In this instance I created a new dedicated “NAS” application for storing the test data.

Software configuration was pretty simple.  A number of screens requested the SDN Account details and an activation key for the CloudNAS product, then the final screen sets the Drive letter and installation directory.  Once complete and the CloudNAS service started, then the virtual drive (in my case X:) can be shared out.

For my testing I chose to use a configuration mode which writes data to a local cache before uploading to the SDN.  This ensures data is available if the server crashes or is rebooted.  It’s also possible to run in a mode where data is written directly to the SDN as it is written to the virtual drive.  Clearly this is more risky as any data not uploaded would be lost if the server crashed or was rebooted.

In testing, the CloudNAS application seemed to perform well, however  I was only able to do limited performance testing and couldn’t generate a full scalability test with multiple clients.  As data was written to the virtual drive, I was able to monitor the progress via the CloudNAS Monitor tool, which runs on the server.

Under the Hood

I always like to have a little dig around when I install software.  I’m keen to see how it has been written and if certain standards of design have been thought through.  These standards include scalability, performance and usability.  Unfortunately, CloudNAS leaves me with some concerns.

The first thing I noticed was the use of a virtual file system tool called Dokan.  This appears to be the way the SDN CloudNAS file system is emulated under Windows.  Whilst using third-party plug-ins is no bad thing, in this instance it doesn’t appear that Dokan is a commercially supported product.  I would have grave reservations about placing my data under a tool that was dependent on best-efforts home-grown software.

Usability throughout the product is woefully inadequate.  Changes to the configuration parameters requires effectively stepping through the installation process.  The “upload monitor” is a tool running on the server itself and doesn’t even auto refresh.  Probably most disappointing is the lack of a browsable interface into both the local view of the filesystem compared to that in the cloud, with an indication of differences for data not yet uploaded.  

I could find no way to list the files and volume of data already moved up to the cloud.

CloudNAS will not be suitable for certain types of files; data is only uploaded to the cloud when files are closed; there’s no file locking (presumably due to the Dokan implementation).  This could significantly restrict the benefits.

CloudNAS only presents a single virtual drive.  There also doesn’t appear to be any method for moving the local cache to another location (for example on a separate disk).

CloudNAS appears to be a direct Linux port, without Windows design thoughts.  Logging is to flat files which have to be browsed in their target directory.  Configuration parameters have to be edited directly in *.conf files.  The most telling clue is the lack of full conversion of the user manual to Windows, with sloppy references to Unix style directory names, like /var/cache/nirvanix.

Summary

In summary, I’d say what could be a promising product is let down by poor implementation.  The whole user experience hasn’t been thought through fully, leaving the product feeling like a University programming project. A major overhaul is needed.  This should include proper Windows menu driven options settings, an automated refreshing console which displays the local and cloud data structures, plus basics like an indication of the volume of data stored online.  The product needs to be completely refocused on the user experience.  After all, the move to cloud storage is about delivering a service – the back-end technology will be expected to just work;  what will become important in the future as cloud services mature, is usability – unfortunately sadly lacking in this version of the product.

Now, don’t forget I’m a fan of the concept of cloud storage, but I also think that we need better data availability models to cope with the failure of a single provider (I wrote a series of posts on this starting here).  However, reading the comments on the TechCrunch site, one entry from Chris Walker made me think that people aren’t understanding the way IT works:

“I believe data stores in S3 auto-replicates across multiple data centers so theoretically you should not encounter a data loss like this one – unless all of amazons data centers get nuked.”

Data loss is not purely a hardware failure issue.  Information is lost for many reasons; whether that’s malicious data destruction, lack of forethought on IT design, a rolling disaster, failure in replication, or something else, you can’t assume just because data exists in multiple locations that it will be 100% safe.  

Data security is as much about process as it is the technology.  Here’s a few questions to ask:

• Is my data replicated in real-time (so if I delete/corrupt it, I ruin all copies)?
• Is my data saved in snapshots (helps prevent the above issue)?
• Is my data secured on multiple technology platforms (e.g. primary, secondary and/or tape storage)?
• How is my data being replicated (tools and process)?
• How can I prove my data has been backed up/replicated successfully?
• What level of user/electronic/physical security is in place?

Unfortunately, the appeal of online storage services today is the low cost and ease of use.  Perhaps we should also be giving more weight and consideration to the value of our data before we commit to using these services; it may then be worth paying a little more to get a more resilient service, or to insure against the consequential losses of losing valuable data.