Netapp: The Inflexibility of Flexvols

By On 2 August, 2010 · 44 Comments · In Featured, Storage

I’ve just been reading up on Data ONTAP 8.0 as part of some ongoing work I’m doing.  You may be aware that version 8.0 of the filer operating system brings two major features; 64-bit support and the (sort of) integration of the Spinnaker code to create the multi-node version of ONTAP (called cluster mode).

The move to 64-bit is an essential feature required to enable Netapp filers to scale.  Currently both aggregates and Flexvols are limited to only 16TB.  Plenty of people have complained this limit is far too low and it is one of the most restrictive scaling issues within ONTAP.  By moving to 64-bit aggregates, scalability is increased dramatically (but perhaps not proportially as expected) to 100TB.  Unfortunately things are not all rosy.  For instance, although aggregates move up in size, Flexvols do not; they stay at a mere 16TB.  There are a number of other things to consider that mean Flexvols are still not as flexible as they should be.  For instance:

  • There is no option to move Flexvols between aggregates without taking an outage.  Volumes can be copied (vol copy command) or cloned (vol clone command).  As volumes get larger, taking extended outages to simply move data is unacceptable.  Netapp needs to add the facility to transparently move the underlying location of a Flexvol without user impact.
  • Aggregate type (32-bit or 64-bit) is determined at creation time.  This means a 32-bit aggregate, once created, cannot be expanded past 16TB.  It also means that after a migration of an existing system to Data ONTAP 8.0, the aggregates can still not be expanded.  If you’re hoping an upgrade will give you extra flexibility – it won’t.  You will have to create new aggregates and migrate your data – which of course can’t be achieved without outage, as we’ve just discussed.
  • A system with 64-bit aggregates cannot be reverted to a version of Data ONTAP lower than 8.0.  So, think long and hard after that upgrade about whether you need 64-bit aggregates.  Once you create them, they are here to stay.
  • Aggregates can be increased in size by adding disk – but not reduced in size.  So, there’s no flexibility in being able to temporarily increase an aggregate size, then reducing it when capacity requirements decrease (or when data has been moved elsewhere).

It’s disappointing that there are still significant restrictions in the use of so-called FlexVols with the new release of Data ONTAP 8.0  Fracturing the code into “7-mode” and “cluster-mode” doesn’t help – for instance the new vSphere VAAI extensions are not supported in Cluster Mode, so this isn’t a practical route to take in order to get the additional functionality.  With the noise we hear from Netapp about virtualisation, the address space for the number and size of objects should be much larger than you wil never need.  Unfortunately we are still pushing the logical limits which causes issues with utilisation and mobility, increasing operational cost.

On balance, I haven’t reviewed other NAS vendor’s products to do a full comparision here, however as Netapp are perceived to be the market leader, I’d expect more from them after nearly 20 years of development.  There are other NAS products out there and I can’t help thinking that it’s only a matter of time before their market share starts to hit the Netapp bottom line.

Tagged with: • • • • •
 
If you enjoyed this article, please consider sharing it!
Reddit Facebook Twitter Delicious Mixx StumbleUpon Digg

  • Pingback: Tweets that mention The Storage Architect » Blog Archive » Netapp: The Inflexibility of Flexvols -- Topsy.com

  • http://www.storagewithoutborders.com John Martin

    Chris,
    I understand your dissapointment, there are things I’d love to have seen included in 8.0 that got pushed into subsequent releases, but the transition from Data ONTAP 7 is an important change to the underlying architecture, and its important that we we are 100% confident that those changes are all for the good.

    having said that there are a few factual errors in your assertions. I did put a long a detailed reply, bu the captcha code had cookie problems and it got lost, and I didnt feel like spending another hour re-writing it. The major error from my perspective had to do with non-disruptive migration of workloads. You can move volumes between aggregates using snapmirror migrate and also via “Data Motion”, they have their limitations, but they are being used at large customers today, especially for NFS and iSCSI workloads.

    Also with regards to not being able to remove disks from aggregates, its not something I’ve really heard as being a problem before. For the most part, you really dont want to be allocating and deallocating physical objects. In an ideal world you’d place all your physical storage objects into one big aggregate and share them amongst an arbitrary number of controllers and carve out the IOPS capacity, and reliabilty into flexible logical storage entities. We’re not there yet, but 64bit aggregates are a good beginning.

    The thing about cluster mode in ontap 8.0 and 64bit aggregates is that for most people they’re interesting but irrelevant to their needs. The number of workloads that need a scale out storage solution for HPC environments provided by cluster mode, or large 100+TB flat namespaces provided by 64bit aggregates is a vanishingly small percentage of the tens of thousands of netapp customers world wide. We provided this functionality so that early adopters who needed to use, or get familiar with the technology could do so quickly. For a large number of customers 7 mode will remain the preferred deployment option until they become comfortable with cluster mode which they will do in their own good time.

    Some people, especially those pushing the limits of technology do need these limitations removed sooner rather than later. If you’re one of them, I’d strongly suggest you get a briefing on whats coming out in 8.01 (real soon now), and 8.1. I think you’ll be pleasantly surprised.

    Regards
    John Martin
    Principal Technologist
    NetApp ANZ

  • http://www.brookend.com Chris Evans

    John

    Thanks for the comments and I apologise for your problems with the captcha code. I will check out Snapmirror migrate and Data Motion as you’ve mentioned.

    The ability to remove drives from an aggregate gives the flexibility to replace one drive capacity with another without having to build a new aggregate and migrate data. I’d rather treat an aggregate as a pool I can expand and contract as required in order to perform these kinds of tasks. My point was, as aggregates scale up to 100TB, I don’t want to be performing massive migrations in order to simply remove a RAID set. I suspect customers haven’t asked for it today because they haven’t hit the problem (yet). As 100TB aggregates arrive, no-one will (a) want to migrate that volume of data unnecessarily (b) want to purchase additional 100TB of capacity simply to re-organise their disk layout.

    So, my point is that the current aggregate & flexvol features don’t cater well for highly scaling environments.

    I’ll check out the snapmirror/data motion you’ve referenced and post on it as an update.

    Cheers
    Chris

  • Ryan B

    A few years back I migrated from EMC Clariion to NetApp and was shocked to find that online LUN migrations were not possible in ONTAP. This has been an enormous obstacle for us – especially when coupled with the 16 TB aggregate size limit. It has crippled our ability to effectively manage data throughout its lifecycle.

    My understanding of Data Motion is that it operates on vFilers, which makes it almost useless as a data migration tool since there is no volume level granularity. Oh, and vFilers don’t support FCP.

    snapmirror migrate does not non-disruptively migrate LUNs.

  • http://www.brookend.com Chris Evans

    Ryan

    Thanks for the comments. It seems to me that snapmirror migrate and Data Motion are sticking plasters to solve the data mobility issue, rather than built in features of the code. It makes you wonder how long Netapp can continue to add features in this way.

    Chris

  • http://www.storagewithoutborders.com John Martin

    I’m not sure about what you mean by “not built into the code”, the core functionality has been part of OnTAP for what must be close to 10 years, most of the recent changes have been around reducing te number of steps involved in making it easier to use. As far as I know there isnt anything else out there that allows you to move an iSCSI LUN or NFS datastore non-disruptively from one array to another including it’s IP address, VLAN settings and security context. Nonetheless perception is reality and based on you’re other posts it seems like you have an issue with NetApp and are likely to view most of our technology from a fairly negative position. Maybe this filters your perception.

    I’m also unclear why the use case is for removing a RAID group from an aggregate is so important to you. Is reducing te performance and capacity of the aggregate something you think needs to be done on a semi-regular basis ? On an aggregated basis, I/O and capacity demands tend to increase over time, though the specific logical entities that make up the aggregate will grow and shrink. This is the logic behind the ability to non-disruptively add performance and capacity to an aggregate (generally in 5 disk increments) and dynamically grow/shrink/QOS/dedupe/protect the logical entities (the flexvols) that sit on top of it.

    Regards
    John Martin

  • Ryan B

    John,

    Maybe we could step back to the title of the post: “Netapp: The Inflexibility of Flexvols”. A specific concern raised by Chris was having “no option to move Flexvols between aggregates without taking an outage”. Your point about moving “an iSCSI LUN or NFS datastore non-disruptively from one array to another” is not at issue here.

    The problem NetApp can solve is data mobility within an array. EMC provides for on online migration of LUNs within the Clariion (FCP and iSCSI), and NetApp should offer the same functionality. With FCP LUNs it should even be possible to migrate between controllers in a NetApp cluster non-disruptively…

  • http://www.brookend.com Chris Evans

    Ryan

    That’s part of my point – data mobility is a key requirement especially a environments scale and we start considering PB arrays. No-one wants to manually move data to balance workload, physical capacity etc. Let’s be honest, if Netapp offered infinitely scalable multi-tier aggregates with block-level tiering this whole discussion would be moot. The point is they don’t; there are restrictions which manifest themselves as operational overhead. I would want features that minimise those overheads whilst not introducing others – like mass migrations to move a volume.

    Chris

  • http://www.recoverymonkey.org Dimitris Krekoukias

    Hi all, Dimitris from NetApp here.

    With any system there are limitations and tradeoffs based on the underlying design. Some things are easier for NetApp that are very hard for anyone else – and vice versa.

    I can write a similar post lambasting other vendors and asking why they don’t all do the things that ARE NetApp-exclusive.

    See here: http://bit.ly/dmJrcA and http://bit.ly/anPWCw and http://bit.ly/dsj1sS

    The design of every system lends itself better to some things than others.

    If all you want to do is dynamic LUN migration, then look elsewhere.

    If you care about a robust, mature, high-performance, TRULY Unified system (all storage protocols including FCoE) with application-aware snapshots and replication and features, deduplication for any protocol, advanced huge caches and more, then NetApp is the ticket.

    BTW a FlexVol in 8.x can be the size of the aggregate, so not 16TB. The 16TB limit (so far) will remain for a deduplicated vol.

    Dynamic LUN migration is, indeed, a good (nay, great) thing to have. But it’s not the only important element.

    And, finally, assuming certain features will never come is misguided at best. I can’t say more.

    D

  • http://www.netapp.com/ Uday Boppana

    Chris,

    You have some good thoughts/ideas about the future needs and wants of the storage market, some of which seem to resonate with the features NetApp is offering.

    Some clarification on the blog post above though:

    - The maximum size of a FlexVol volume in Data ONTAP 8.0 is not limited to 16 TB. It can be same as the aggregate size limits, which is up to 100TB.

    - As for data migration from 32- to 64-bit aggregates, Data ONTAP 8.0 and 64-bit aggregates are only the first step in the direction of providing extreme scalability and non-disruptive operation. Currently, data migration is required to move data from volumes in 32-bit aggregates to volumes in 64-bit aggregates; however, future versions of ONTAP will build on this base to increase scalability and also make the related process non-disruptive and seamless.

    - Among NetApp users, there is really no precedent for reverting to older-generation releases. Our customer base currently using Data ONTAP 8.0 has been highly satisfied with the stability and functionality of this release. As with any newer version of software, reverting to older version requires disabling/destroying the new features that came with the new version. However, it is highly unlikely that someone running Data ONTAP 8.0 wants to revert to an older release.

    - FlexVol volumes can be dynamically resized – they can be both increased and decreased non-disruptively. Thin provisioning enables you to over-provision an aggregate by creating bigger FlexVol volumes than available storage space in the aggregate. Aggregates are the physical layer of storage, but the actual storage containers from a client’s perspective are FlexVol volumes. It is best to place disks in as few aggregates as possible and then resize FlexVol volumes to suit data allocation needs. FlexVol volumes can be expanded or contracted, depending on the need, without having to worry about the physical disks that hold the data. Given this flexibility with FlexVol volumes, removing disks from an aggregate is not relevant to easing data management.

    Another important to consider with NetApp systems and Data ONTAP is that we offer a unified multi-protocol capable storage system. So, when considering a NetApp system it is imperative to evaluate it from an overall data storage and data manageability aspect rather than look at limited SAN or NAS functionality. Some of the traditional SAN or NAS issues or concerns might not even be relevant when evaluating a NetApp system due to this unified multi-protocol architecture. Our unified storage offers some of the best SAN and NAS functionality and even more because it is built on unified storage architecture.

    Uday Boppana
    Product Manager, WAFL
    NetApp

  • http://www.brookend.com Chris Evans

    Dimitris

    A quick scan on Google shows that you do indeed write posts lambasting other vendors, mostly EMC from what I can see.

    I believe you’re missing my point. Netapp like to see themselves as the market leader in NAS (forget the Unified Storage view, as that’s not relevant) and I believe as a company they are the leader by market share. However, as the encumbent I’d expect Netapp to be leading the way in innovation, operational improvements and so on. I don’t see that and the improvements in aggregate/flexvol have only just arrived with version 8.0 – which forks the code into two new versions – so now we have Data ONTAP classic, GX and cluster mode. I’d have expected more earlier than this – many many other NAS vendors have supported much larger address spaces already.

    As for the size of volumes, unfortunately the documentation isn’t clear or precise – I check my facts as thoroughly as possible before publishing; The Storage Management Guide indicates the size of a volume is system dependent; the System Configuration Guides for the latest models only shows traditional volume size – not flexvol size. Clearly more work required by the Netapp technical authors there.

    No doubt Netapp have certain features in the pipeline, however with the current history of feature and code integration, I expect customers won’t be holding their breath to see any of them soon.

    Chris

  • http://www.brookend.com Chris Evans

    Uday

    Thanks for your response. As I mention in reply to Dimitris, your documentation is not clear on flexvol size and in fact, I don’t see why model type should have an infuence on it if flexvols are truly virtual volumes.

    That aside, I understand entirely the scenario of working with aggregates and flexvols. In fact, I believe Netapp introduced aggregates because the rapid increase in disk capacity meant adding a RAID set to an existing volume was an expensive business.

    Reducing the number of aggregates to as low a number as possible doesn’t provide management flexibility. You’re constraining the customer into a “one size fits all” scenario. There may be many reasons to want to change the configuration of a filer; from technical, performance, administrative, billing and so on.

    This all gets back to my original point – flexibility, options for managing large volumes of data non-disruptively.

    Hopefully we’ll see those new features to which you refer coming out soon. I’ll be watching to make sure you don’t sneak in the ability to remove RAID groups from an aggregate….

    Chris

  • http://www.grumpystorage.com ianhf

    Chris,

    Great post, and thanks for reminding me of the topic! And good to see NetApp people joining in positively with comments.

    I’ve a bunch of thoughts that I’ve scribbled out here on this area http://www.grumpystorage.com/2010/08/notapp-random-thoughts.html – certainly agree there is work needed on this space.

    NetApp – you need to do a customer council session, and Chris needs to be on the invite list, and we’ll give you clear feedback on roadmaps, current situations, RFEs, issues etc. Time to get the recorders & notepads out to listen & act in Europe rather than daydream in sunny California!

    Cheers

    Ian

  • Pingback: The Storage Architect » Blog Archive » Netapp: The Inflexibility of Data ONTAP

  • http://storagebod.typepad.com Martin G

    Actually, one thing which I must take issue with is the constant claiming that NetApp support all storage protocols. Even excluding some of the more obscure ones…they are missing what is still an important protocol for many, that is FICON.

    And for me, they don’t support Appletalk either, which is a bit of pain in media land.

    I think it is fairer to say that NetApp have produced a Unified Storage Platform for all protocols that NetApp support but NetApp doesn’t support all storage protocols.

  • http://www.brookend.com Chris Evans

    Martin

    Hmm, FBA architecture with CKD data – interesting thought. I’d agree with you that the term “unified” has been massaged to mean basic SAN/NAS protocols, i.e. block-based and file-based. Maybe we need some new term; Partial Support of Multiple Standards? Semi-unified, part-unified; I can’t think of one.

    Chris

  • http://chucksblog.emc.com Chuck Hollis

    Chris, Martin

    Lively discussion, and it’s nice to see the EMC people refraining from jumping in with both feet.

    Chris probably knows the the Symmetrix concurrently does the z/OS protocols (FICON, ESCON) all the FC and iSCSI stuff, as well as support for interesting edge cases like iSeries, Bull, Fujitsu, Teradata and even some support for things like VAXcluster and HP3000s if I remember right :-)

    Maybe we should have called it “unified storage” …

    – Chuck

  • Karl Dohm

    I’d like to relate one case I ran into where the capability to remove disks from an existing aggregate seemed quite important.

    I once had the goal of following NetApp recommendations to maximize space efficiency, specifically in this case by optimizing the use of aggr0 (aggregate 0). Aggr0 holds the root volume for a filer – its persistent store for the operating environment needed to manage the filer. By default this aggregate comes factory configured as RAID-DP containing 3 spindles, meaning only one is truly holding data and two are present for parity. In order to better optimize use of these two parity spindles (and the rest of the data spindle), NetApp sometimes recommends expanding aggr0 and adding customer volumes to it.

    That sounded reasonable to me so I went down this path. But in using aggr0, I ran into a major pitfall that was exasperated by the inability to remove spindles from an aggregate.

    Using Filerview, I increased the spindle count for aggr0 from its size of 3 to 20 spindles.

    After having done this, I had the realization that the raid group size of aggr0 was by default set to 16. Unfortunately I wanted to work with a raid group size of 20. When looking at raid group details of aggr0, it contained one raid group of size 16 and another of size 4. 4 of those 20 spindles are not contributing to read throughput, so to me this configuration was not acceptable. The question then became, how could this be turned into an aggr0 with raid group size of 20? Answer is – it can’t – without reinitializing the filer.

    A FAS aggregate allows modification of its raid group size, but once the spindles are inserted, you are stuck with whatever the raid group size was when you did the insert. In other words, had I realized the raid group size was 16 prior to adding the 17 additional spindles, I could have changed the raid group size to 20 and the aggregate would have added the 17 spindles to create a single raid group with 20 members. But since I made the mistake of adding the 17 spindles then realizing the problem (the default setting), I was in a pickle.

    If I could have removed those 17 spindles from aggr0, then reset the raid group size to 20, then re-added them, I could have avoided the ensuing mess.

    In order to undo this and change the raid group size of aggr0 to 20, the entire filer had to be reinitialized. That meant the brains of the filer were completely reset – all data scrubbed, all configuration settings lost. The initialization itself was a multi-step, complex process which took me several days and required the involvement of NetApp support. I was lucky enough to be able to find an extended time slot when this filer did not need to be actively serving any data. I’m not sure what a non-stop shop would do. Perhaps they would just live with the raid group size being 16 instead of 20 in order to avoid an outage.

    Undoubtedly a more skilled NetApp administrator could have reset the array more quickly, but for certain NetApp does not make it easy to undo this rather simple mistake. The bigger point is, though, that the inflexibility of FAS aggregate management can come back to bite you in unexpected ways.

    Karl Dohm
    Storage Architect
    Hewlett Packard

  • Matthew Emery

    I am a longtime Netapp Administrator and agree that moving volumes between aggregates is a pain.

    Technically its pretty easy, the hard bit is arranging the outage with the business. The outage is normally only a minute or so, but it still needs to be done in an outage window, and that’s always a pain to organise.

    A large part of the pain I see is due to the inflexibility with the CIFS protocol, as its persistent, even a small outage to the filer (such as a cluster failover or volume migration) can mean clients disconnect. NFS is much easier to deal with.

    As far as Karl’s problem with the aggregate raid group sizing, you made a Netapp Newbie mistake, easy enough to do I suppose if you haven’t worked with a Netapp before. Also you don’t have to reinitialise your Filer to get your aggr0 back, you just have to move vol0 (the configuration volume) to another aggregate, and then destroy aggr0. That still requires a small outage (a reboot), but doesn’t need days of downtime.

    The point here, Netapp does have some issues with migrations, but also, read the doco.

    Cheers

    Matt

  • http://p2v.it/2010/07/26/simple-way-to-extend-an-aggregate-in-a-netapp-fas/ Radek

    Chris,

    Can you please publish your NetApp credentials? Although I see several lucid blogs from you regarding legacy SAN array issues, I have to say your comments regarding NetApp storage functionality appear uninformed at best, ignorant at worst. Having infamous EMC and HP bloggers unnecessarily fan these false flames via ignorant comments only confirms my suspicions.

    Regardless, a quick scan of NetApp product documentation on NOW.NETAPP.COM instantly reveals perfectly correct information which you seemingly missed or overlooked, such as:

    “How 32-bit and 64-bit volumes differ
    32-bit volumes have a maximum size of 16 TB. The maximum size of 64-bit volumes is determined by the size of their containing aggregate—up to 100 TB, depending on the storage system model.”

    Working on several large storage projects for a major systems integrator, I appreciate the fact NetApp has made the upgrade to ONTAP 8 nearly painless for 7G installations. I prefer to setup new volumes under ONTAP 8 and have my prior 7G versions upgrade smoothly. Sun created a lot of heartburn in the past by attempting to accelerate the premature cutover to Solaris from SunOS. Microsoft’s prolonged approach to NT migration proved more prudent in hindsight and I’m glad NetApp is learning those same lessons. You can’t switch a giant installed-base running business-critical systems to a radical new architecture overnight.

    FInally after working on ONTAP 8, I’m surprised you also failed to mention the most important management concept in ONTAP 8 c-mode, which is the Vserver. Implicitly, all of its operations (such as growth, shrinkage, load-balancing, read-mirroring and of course volume migration) are non-disruptive. This bodes very well for the future c-mode protocol support promised in 8.1, although I hear plenty of goodies are on their way even sooner in 8.0.1

    Compared to RAID group management in other arrays, I find NetApp to be much simpler and more robust, because you never have to trade off efficiency (RAID 5) for performance (RAID 10) or availability (RAID 6). RAID-DP quite simply has it all. Here’s a recent example which explains it:

    http://p2v.it/2010/07/26/simple-way-to-extend-an-aggregate-in-a-netapp-fas/

    I think you should take some official NetApp training, then get some true hands-on experience before commenting further on NetApp technology. Anything less will further tarnish your credibility. I believe once you fully understand their technology (rather than merely opining on it) they would gladly welcome you to a proper feedback session.

    I’d be happy to facilitate any introductions you may need to a local NetApp technical representative.

    Radek

  • http://www.brookend.com Chris Evans

    Radek

    I have been using Data ONTAP (then version 6) since 2002, the same year I did all of the Netapp training. I’ve since seen the development of aggregates as a feature and even today I am working with a global customer that has hundreds of filers, trying to identify ways to improve their utilisation. So, I can demonstrate credibility; something perhaps you should have chosen to do before being so condescending in your comments. You make no effort to identify why you are such an expert on Netapp technology.

    Let’s look at the rest of your comments, shall we? Firstly your assertion on the Flexvol size – the comment states that new Flexvol size is dependent on model. Check the System Configuration Guide for most common systems and you’ll see no size is quoted (only aggregates and traditional volumes), so to be pedantic, Netapp haven’t quoted a specific size for 64-bit aggregates in their documentation. I’m not so naive as to post a comment without having done my homework.

    As someone who has clearly drunk the Netapp koolaid, you have been suckered into the standard vendor promise of all the features you want being “in the next release”. Perhaps you should take some time and look at the following blog entry http://www.grumpystorage.com/2010/08/notapp-random-thoughts.html#comments – where some of these things are discussed further.

    You clearly can’t see the “big picture” being discussed here and are getting bogged down with technicalities. This blog is called “The Storage Architect” – not ” The Storage Administrator”. My concern is the wider ramifications if designing and implementing storage systems on large scale, something I have done for many customers over many years.

    So rather than indulging in rather immature cheap personal jibes, why don’t you add something useful to the discussion – perhaps explain why you feel setting up new installations rather than migrating existing ones is more preferable, for example. I suspect because you’re not prepared to admit the flaws in migration are really there.

    In this instance I’ve chosen to publish your comment; I’d consider offering more courteous ones in the future if you want those to appear on this site.

  • http://www.brookend.com Chris Evans

    Matthew

    Your point about CIFS is an interesting one; assuming we are not migrating “out-of-filer” then surely the CIFS and NFS sessions are all handled by the same physical hardware, so moving a volume between aggregates isn’t involved in this process; data could be read from the source aggregate and as we know, all writes are new in place, so therefore could be written to the new aggregate as they occur. In the background an additional task could be migrating inactive blocks of data to bring the volume to its new location. This doesn’t strike me as a difficult thing to achieve and in fact I commented on something similar last year in this post http://www.thestoragearchitect.com/2009/10/21/enteprise-computing-automated-tiering-why-move-the-data/

    I agree, moving to a new filer does present problems for CIFS and that’s a separate problem that perhaps is best approached with other technology sitting above the filer.

  • Matthew Emery

    Chris,

    The problem with migrating CIFS and NFS is that the share or export name is tied to the volume, so that in order to guarantee that you have captured all writes you need to remove the shares, do a final snapmirror then recreate the shares on the new volume, and thus you have an outage.

    To be able to virtualise the shares, even temporarily for the final part of the volume migration, so that writes can be buffered or something similar, would be a wonderful thing. It would be one less thing that Change Control can reject my changes for.

    I can dream …. or migrate to Ontap 8 (c-mode of course).

    Cheers

    Matt

  • http://www.brookend.com Chris Evans

    Matt, agreed I think having completely virtual volume names would have been ideal. I have more work to do in looking at cluster mode. I’m hoping there are some good features in there. As usual, the migration will be the challenge.

  • Matthew Emery

    Hey Chris

    I agree that migration will be a challenge, the migration from trad vols to flexvols took a long time for some businesses, and I am sure the same will apply from 7-mode to c-mode.

    I am looking forward to good migration strategies being designed so we can deploy cluster mode with limited impact to the business.

    But, as always, the devil is in the detail, and good critical review of such strategies is important in the storage world.

    Cheers

    Matt

  • Radek

    Chris,

    You’ll need to develop much thicker skin if you want to post vendor half-truths as fact to bait haters and fanbois into a low-value discussion.

    We seem to have different approaches to providing infrastructure architecture consulting to our clients. I prefer to maintain discreet, professional and cordial relationships with all my vendors, both major and minor. This affords me privileged access to their subject-matter experts on a regular basis which takes the guesswork out of long-term planning for my clients.

    As you no doubt know, large enterprises do not have monolithic requirements. Technology refresh occurs on a staggered basis meaning that often many (if not most) projects do not need next-generation features to be successful today. Consequently, vendor roadmaps (such as NetApp’s) are often ahead of most real-world enterprise requirements. My value is mapping the client’s current and projected infrastructure needs to the best available technology on a long-term basis. Ever since NetApp introduced 7G, they have fit the bill as the storage component in support of the majority of those projects (Exchange, SQL, SharePoint, VMware, Oracle, SAP)

    I find it very revealing and not too surprising that the most critical bloggers online tend to be the most starved for current and accurate product information.

    You may call that drinking the Kool-Aid, but I prefer to think of it as professional services differentiation.

    I wish you the best of luck in your current ONTAP 8 project!

    Radek

  • Karl Dohm

    Hi Matt,
    My experience is that you have to do an awful lot of reading (and reading between the lines) in netapp documentation to tease out this quirky behavior. It seems avoidable – cant the GUI guide the user into not falling into traps?

    You mention moving vol0 to another aggregate to solve this problem, but I’m not following how this is done. Can you describe the process? I hope you are not assuming that all users have CIFS or NFS licenses.

    Karl

  • http://www.brookend.com Chris Evans

    Radek

    Without a doubt that’s the best excuse I’ve heard for vendor non-delivery:

    “Consequently, vendor roadmaps (such as NetApp’s) are often ahead of most real-world enterprise requirements.”

    Come on, get real. Large enterprises are stretching Netapp’s technology and they’re not keeping up.

    If you’re content to continue supporting a single vendor, rather than offering your customers an objective opinion, then good for you. I will continue to applaude the good and criticise the FUD. Objectivity and independence is a hook I’m happy to hang my hat on – one you’re clearly not.

    Chris

  • Matthew Emery

    Karl

    The Netapp GUI is not a good tool, most netapp admins that I know only use the command line. Personally the only thing I use it for is to add drives to aggregates, as it gives you a Confirmation box before you apply the change, which you dont get by running the aggr add command on the filer.

    As disk adds to aggregates are irreversible, I would tend to agree that more “here is what is going to happen, are you sure?” type messages would be beneficial to less experienced admins.

    A quick overview of how to move vol0 (there are detailed howtos available if you google “move vol0 netapp”) you dont need CIFS, or NFS. (side point you can access vol0 without a CIFS license via CIFS if need be – run cifs setup on the filer.)

    create new vol0 on different aggregate with a name like vol0_new
    copy current vol0 to vol0_new using snapmirror or volcopy or ndmpcopy
    run command vol options vol0_new root (marks new vol0 as the root vol)
    reboot (or cluster failover / failback if you want to do it safely.

    rename old vol0 to vol0_old
    rename vol0_new to vol0

    done.

    you can then destroy the aggregate and rebuild it.

    Chris – I hope using your blog for Ontap Geekery is ok.

    Cheers

    Matt

  • http://blogs.netapp.com/getvirtical Peter Learmonth

    @Karl
    There’s another solution for you to increase RAID size, although it doesn’t fix/convert your initial 16+4 config. You can change the raid group size with the “aggr options raidsize xx” command, which affects any disks added after that command. As you add disks, you can specify into which rg they go with the -g option to aggr add, for example “aggr add aggr0 -g rg0 4″. This is in the man page for aggr and the Storage Management Guide.

    Re: 16TB vol limit
    Others have already corrected this, but here’s a simple little “demo” I did for another request showing it in action, along with VMware actually using it.
    http://tinyurl.com/BigVols

    Share and enjoy!
    Peter

  • http://www.recoverymonkey.org Dimitris Krekoukias

    I posted a response to the aggregate issue here: http://bit.ly/duBafj

  • Pingback: The Storage Architect » Blog Archive » Mad, Bad and Dangerous to Know

  • Karl Dohm

    Thanks Matt,
    Yep, I realized a long time ago that the FAS is CLUI centric. I prefer using filerview for some activities though since it keeps me from having to remember syntax. I’m not in there every day!

    Isn’t there a risk in the root volume changing in some critical way during the middle of this process? I.e., after root_new is created, some important change happens to root, meaning that change is lost. I’m assuming the use of volcopy or ndmpcopy, not snapmirror (which typically involves two expensive optional licenses).

    Obviously the goal is that the vol0 move needs to be done safely without additional licenses – if this is something the typical admin should be doing to correct the raid config.

    Peter, thanks for chiming in. Yes, I realized that, but for me it wasn’t good enough to have one raid group of size 16 and the rest of size 20 in that aggregate. In fact (at the time) I wanted one aggregate with exactly 20 disks – all in the same raid group.

    Karl

  • http://www.recoverymonkey.org Dimitris Krekoukias

    Hi Chris,

    Since you are trying to help customers that already have NetApp, here’s an idea: It might be easier to have a meeting with a NetApp engineer that, after having you sign an NDA probably, will be able to answer all your questions. It will save you a lot of time and will probably change the tone of your posts.

    I notice at the moment you’re going to various websites looking up docs etc – there’s altogether too much documentation and I admit certain things aren’t covered in the docs you’d expect them to be (but may be in others).

    So, do yourself a favor and start a relationship with someone from engineering… :)

    I’ll cross-post that in your other entries.

    Thx

    D

  • Radek Kubka

    Guys,

    A good friend of mine called me today mentioning this (rather fierce) discussion. To my surprise he has automatically assumed ‘Radek’ equals ‘Radek Kubka’ – turns out I am a ‘recognizable’ individual amongst Storage/NetApp community. So simply to make things clear I am saying this firmly: I am not the same Radek as the one who put his original comment few screens above.

    Thanks,
    Radek Kubka

  • Pingback: Client-server : Computer Industries Newest and Hottest Buzzwords-business

  • Pingback: The Storage Architect » Blog Archive » EMC Delays New CLARiiON and Celerra?

  • Pingback: What Should We Look Out For From a Shared Web Hosting? | shared web hosting

  • Chris

    I ran across this post and thought I’d add an additional detail which makes NetApp extremely unattractive if you wish to create large LUNs, ie. bigger then say 16TB.

    Even within their latest release 8.0.1 they do not support creation of LUNs larger then 16TB. Sure you can create a FlexVol which can be up to 100TB dependent upon your controller platform but that 80TB (gotta account for snaps) volume can only be used for serving out NAS shares.

  • Scott

    Also just came across this thread. Couldn’t resist getting involved:

    First and foremost, I would point out that of the bullet points above, only one of them (Non-Disruptive Migrations) has anything to do with the FlexVol itself. The rest are complaints about the inflexibility of Aggregates. That is a completely different animal.

    There is a *major* difference between the flexibility of FlexVols and the flexibility of Aggregates. NetApp Volumes are inherently flexible things, as they are virtualized. Aggregates are inherently inflexible things, as they are striped sets of RAID groups. All of the inflexibilities of Aggregates can be attributed to this fundamental RAID building block.

    I’m not aware of any storage vendor today who can haphazardly change the number of spindles in a RAID GROUP. Honestly, I would be wary of any vendor claiming they could do this. The RAID array is the basis of your data protection. It is the inflexibility of RAID that makes it safer. Vendors must find ways around the inflexibility of RAID by abstracting the volumes or LUNs. Non-disruptive LUN/Volume migration is just one of many possible advantages of this abstraction.

    The number of flexible things NetApp *can* do with a volume are too many to list here. Thin provisioning, block-level dedupe on your active storage, and dynamic volume resizing are just a few of them. More importantly, Non-Disruptive LUN/Volume migration is available today. Data OnTap 8.01 includes this feature, and it became available about 1 or 2 months after this thread was started. Of all the FlexVol and Aggregate “inflexibilites” listed, I would say this was the only one that compared negatively with other storage vendors. But that issue is gone now.

    @Chris (recent post): You are right about the 16TB LUN size limit being an issue. Most of my customers don’t have operating systems that can take advantage of larger LUNs, but my Mac customers certainly can. And this limit forces them to use LUN aggregation tools. I would point out, however, that your volume size can be as high as 100TB. Just turn snapshots off if you don’t want instant recoverability of your data. ;)

  • Greg McAllister

    thanks for the interesting read… been away from storage admin for a couple years and was looking to learn about what was new at netapp and this was a good read.

  • http://twitter.com/JohnMartinIT John Martin

    Chris – John Martin from NetApp here, I came here to read one of your other articles, and noticed that this post is still one of your most popular, and thought that it might benefit from an update.

    Firstly,for some context. When this post was written NetApp had only just begun to deliver on the ability to move from traditional Data ONTAP that almost all NetApp customers ran (now called 7-mode) to Clustered Data ONTAP (sometimes referred to a cDOT for the sake of brevity). This move required first converging the Spinnaker derived architecture with the Data ONTAP code base, which happened in 8.0, and then completing the integration of the 7-mode functionality into that architecture over subsequent releases along with a bunch of new stuff. This has happened in the subsequent 8.1 and 8.2 releases.

    Given the sizeable base of Netapp customers who get a tad snippy if things go wrong with their storage, this merger, much like the amorous activities of hedgehogs is something that had to be done very very carefully, so there was very little new functionality introduced into 8.0, with larger 64bit aggregates being the only major noticeable difference.Unfortunately this left some people, including you (judging by the content of this blog post) somewhat dissapointed with an apparent lack of progress. Since then however, NetApp has executed on the “Complete” phase of Clustered Data ONTAP and has released two updated versions of ONTAP with significant new functionality which address almost all of the concerns you raised in your post including …

    1. Much larger aggregates (200+TiB)
    2. Much larger FlexVol volumes (50+TiB)
    3, Very much larger Infinite Volumes (10+ PiB)
    4. VAAI Support
    5. Transparent movement of volumes between aggregates

    And a whole stack of other stuff that should result in customers never reaching the logical limits of the system. (The limits I put above are for our mid-range controllers, the high end controller limits are significantly larger again).

    To be sure, there is still some inflexibility in aggregates, they don’t span controllers (though Infinite Volumes do), they’re limited to a few hunreds of TiB, and you can’t shrink them. Then again we don’t call them flexaggrs either, and they aren’t intended to be something you spend a lot of time managing or determining policies for. For most customers, creating one aggregate per disk type will work perfectly.

    FlexVols, on the other hand, are, especially in Clustered Data ONTAP, beautiful software defined storage containers that aren’t tied to physical hardware. They contain data that is accesible via a variety of storage protocols, they are programmable via a number of methods (XML APIs, SOAP, REST, CLI etc), and do a bunch of other cool things too. I know it’s probably because I work at NetApp, but from my perspective there isn’t a more functional logical storage container out there (some come close, but lack, what to me, are important features)

    If you’d like to get a full update on where the limits are of FlexVols/InfiniVols are since the release of Data ONTAP 8.2, and where they’re likely to go over the next couple of releases, let me know and I’ll try to arrange a briefing for you.

    Regards
    John Martin
    @JohnMartinIT

    • http://thestoragearchitect.com/ Chris M Evans

      John, thanks for the post. I think this item remains popular because somehow it has scored high up in search engines. I’ve been meaning to do a review of the details in light of the new platform, from two perspectives; first the “legacy” 7-mode, which at some stage will be discontinued (would be interested to know what the last version of DOT will be) and second to compare the DOT features to those in 7-mode and see what hasn’t cut across.

      I also was in conversations this week with a customer and we were discussing the 7-mode to c-mode migration options. It seems that the options are limited and that’s another area of interest.

      I’ll be sure to drop you a note if there’s anything I need; thanks for the offer.

      Regards
      Chris

      • http://twitter.com/JohnMartinIT John Martin

        Having done a reasonable amount of SEO work for my wife’s business I’ve got a pretty good idea why this particular posts ranks so highly … it might be interesting to crank up some SEO analysis tools and examine where the back-links to this article are coming from ..

        I can understand the concerns around the relatively limited transition options to cDOT 8.1 (For a variety of reasons we refer to the move from 7-mode to cDOT a “Transition” rather than a migration). Part of the reason for this is that clustered Data ONTAP in 8.1 was targeted at new installations, and transitions are intended to be a process guided by NetApp trained personnel. As a result much of the information around that process is restricted to those who have been appropriately trained. (Remember kids … don’t try a major data services transition at home without asking mum or dad first)

        With the completion process being much further along in 8.2, we believe that this is the release where most of our current install base will seriously consider making the transition to cDOT, and the options and tools required to do that are correspondingly much more developed, and will be opened up to a much broader audience. Again if you’re interested I can arrange for you to get a briefing on what that will look like.

        As far as 7-mode goes, it’s reasonable to expect that some customers simply wont want to make the transition no matter how easy we make it, on the grounds that “If it aint broke, dont fix it”. As a result 7-mode support and bug fixes will be around for a very long time, though all the cool new features will go into clustered Data ONTAP.

Subscribe to the Blog!

Subscribe to the Blog!

Don’t forget to subscribe to the blog for more content and monthly emails!