Enterprise Storage Strategies

June 2009 - Posts

• Addressing Enterprise NAS Priorities

  The Enterprise Strategy Group forecasts that the enterprise NAS market will continue to grow like gangbusters over the next two years, reaching 62,000 petabytes in 2012! That number represents a huge sales opportunity for EMC, NetApp, Dell, HP, IBM, BlueArc, and the rest of the traditional NAS array market leaders. It also represents an opportunity for vendors like F5 with their file virtualization systems, as well as cloud storage service providers like Nirvanix with CloudNAS. Jump over to ESG Analyst, Terri McClure's blog to check out the numbers for yourself! What features are enterprise NAS buyers looking for? ESG's numbers show that this continued growth of the installed base of storage systems is a huge problem. So big that the top two priorities for enterprise NAS buyers are the efficiency of all these storage systems from a power, cooling, and space perspective. Simply put, where are the enterprise IT staff going to put another million or so NAS arrays? I have a modest suggestion: Use the following key technologies to reduce the number of systems on your floor! Leverage storage resource management tools to make sure you are using all that NAS capacity efficiently. Hint: Microsoft's File Server Resource Manager software is free with Windows Server 2008 and much improved in Server 2008 R2! Certain data types can be reduced in size using storage optimization, deduplication, and compression tools. Look at Ocarina Networks' Optimizer, for example. Most of the data stored on high-end NAS systems is inactive. Consider using a file virtualization product like the F5 (formerly Acopia) ARX to implement smart storage tiering, moving infrequently-accessed data to lower cost storage systems. Consider moving valuable but less frequently-accessed data to the cloud. Nirvanix' CloudNAS software allows everyday applications to use cloud storage without modification. It makes a great target for your SRM, optimization, or virtualization system! Combining one or more of these options has a massive impact on system efficiency and resource requirements. Consider that nearly every line item in the ESG list can be addressed using these technologies! Posted Jun 30 2009, 10:04 PM by sfoskett with 1 comment(s) Filed under: Microsoft, Ocarina, CloudNAS, Nirvanix, NAS, Acopia, Windows Server, F5, FSRM, ESG

• Cloud Storage, Storage in the Cloud, and Cloudy Storage Systems

  As I discussed in my previous post, What Makes Cloud Storage Different from Traditional SAN and NAS?, today's cloud storage is unique from the SAN and NAS (and even CAS) that has gone before. Beyond the cost and flexibility benefits inherent in public cloud computing resources of all sorts, cloud storage is unique in its openness, programmability, and the possibilities it opens for distribution and collaboration. These compelling benefits, along with an explosion of cloud hype, have led every company with a product even remotely "cloudy" to jump into the market. In my corner of the IT world, everyone from hosting providers to software vendors to traditional array manufacturers are putting forth cloud storage products. As the wise among us already know, these cloud products are not all of equal merit! It has struck me that there are really three kinds of cloud storage offerings today: Storage in the cloud, cloudy storage systems, and full-on cloud storage. Each will likely find its own niche within the overall IT infrastructure landscape, but buyers should be careful when comparing these solutions! Cloud Storage Cloudy Storage Systems Storage in the Cloud Commonly called Public cloud storage Private cloud storage Hosted storage Example Nirvanix Amazon S3 EMC Atmos Cleversafe Amazon EBS Flexiscale storage Capacity Granularity Per-object e.g. $.25/GB/month Per-system e.g. $150,000 for 120 TB Standard config e.g. 20 GB per instance Protocol API* API, file file, block Access Method object/ metadata object/ file file/ block Connectivity Internet WAN/ LAN LAN/ SAN Optimized for Collaboration Flexibility Scalability Flexibility Scalability Familiarity Performance Use case Offsite storage Collaboration Unstructured data Web applications * Note that Nirvanix offers CloudNAS, a standard POSIX filesystem interface Most of the current cloud storage offerings jostling for position in the marketplace fit into one of the categories above, but of course the picture isn't so neat in reality. Some Nirvanix customers, for example, connect to the service through a private WAN or even LAN connection. But these categories can help to cut through the marketing hype and understand just what is being offered by a vendor. One particularly important factor is what I'm calling capacity granularity, which is the basic amount of storage you get and how you pay for it. Consider the diagram above, which shows storage of each of the three types used for a steadily-growing application: If you're relying on hosted storage in the cloud, you probably can't scale like this at all. Your server instance comes with some amount of storage (perhaps 20 GB) and that's all you get. Maybe you can buy another 20 GB or upgrade the server, but flexible scaling isn't the intended purpose of this type of storage. If you deploy your own private cloud, you have to buy some large amount of capacity just to get started, perhaps as much as 120 TB. You then spend the next year filling this capacity up until you need to buy another of these large systems. This model should sound very familiar to anyone who has bought enterprise storage over the last decade! True cloud storage grows with you. You pay for used capacity and can scale as your usage grows. Although smaller organizations like to pay monthly as they grow, large enterprises often negotiate a regular monthly payment. Either way, though, they only pay for what they use! Which cloud storage model fits your needs? Posted Jun 29 2009, 09:05 PM by sfoskett with 5 comment(s) Filed under: Amazon, EMC, Nirvanix, Atmos, API, scalability, Amazon S3, granularity, Flexiscale, flexibility, Amazon EBS

• What Makes Cloud Storage Different from Traditional SAN and NAS?

  Many in the IT industry seem to enjoy arguing exactly what does and does not constitute a cloud service. As I mentioned in my post on the controversy over private cloud services, I do not feel that these arguments are productive. We should focus on results and business value instead of arguing about semantics. However, the current crop of cloud storage solutions have many important differences from traditional SAN and NAS storage, something that seems to surprise many end users I meet. Cloud storage capacity is not your fathers blocks and files! Primary, Secondary, and Tiered Storage Most IT infrastructures contain a wide variety of storage devices, but these have traditionally been divided into two categories: Primary or production storage serves active applications and is accessed randomly. The primary category includes most familiar direct-attached disks (DAS), storage area networks (SAN), and network-attached storage (NAS). Newcomers in the primary category include content-addressable storage (CAS) and cloud storage services, including the Nirvanix Cloud Storage Service. Secondary storage is used for data protection and is normally accessed sequentially. Tape media and optical discs were the traditional secondary storage types, but disk-based systems including virtual tape libraries (VTL) have recently become popular. CAS and cloud systems are also often used for secondary storage due to their lower cost. The performance and capability of primary storage systems varies greatly, as does the price. For this reason, many large organizations classify their primary storage into a number of tiers. Tier 1 storage typically boasts the highest performance, reliability, and cost. Fibre Channel SAN arrays from companies like EMC, HDS, and IBM have dominated this market for over a decade. Most organizations also offer less expensive lower-tier SAN, NAS, and DAS capacity in an effort to reduce their capital equipment cost. Primary Storage Options IT architects are faced with a dizzying variety of primary storage options. Dozens of companies build and sell storage devices, and these leverage a variety of connectivity protocols. Each type of storage presents a trade-off in a number of areas, from performance to cost. There is no intrinsic reason to reject one type or adopt another - the selection process must take into account the technical and business requirements of the application that will use it. DAS SAN NAS CAS Cloud Example Seagate disk Dell PowerVault HDS USP 3PAR InServ NetApp Filer Windows server EMC Centera Caringo CAStor Nirvanix Amazon S3 EMC Atmos Protocol SATA, SAS FC, iSCSI SMB, NFS API, XAM API Access Method block block file/directory object object/ metadata Connectivity Copper cable Fiber optic Ethernet Ethernet Ethernet Ethernet Internet Throughput 1.5 Gb/s- 3.0 Gb/s 1 Gb/s- 10 Gb/s 100 Mb/s- 10 Gb/s 100 Mb/s- 1 Gb/s 1.5 Mb/s- 1 Gb/s Latency 5-10 ms 5-10 ms 20-50 ms 50-100 ms 100-500 ms Use case Bulk storage OS/boot Enterprise applications Unstructured data Archival data Offsite storage Collaboration A large number of options are available for primary storage Enterprise storage technology has evolved a great deal over four decades. The first great step was the separation of the disk from the server in the mid 1960's. Over the next 20 years, protocols were developed to share disk storage among multiple servers, creating the first storage networks. The introduction of RAID in the 1980's led to the development of more virtualized SAN storage systems in the next decade. At the same time, networking companies developed file sharing protocols, creating the NAS market. By the end of the 1990's, the enterprise storage market was divided between block-based SAN and file-based NAS. The limitations of these block- and file-focused paradigms led to the development of content-addressable storage in the first half of this decade. CAS systems discarded traditional protocols and concepts in favor of application-focused APIs and a universal naming standard for unique objects. Many early applications treated CAS objects as simple files. But applications soon developed to take advantage of the capabilities of the unique capabilities of these systems, especially in the document management and archiving space. Enter The Cloud Cloud storage was developed independently from all historical storage concepts, although it might appear to be an evolution of CAS. Both are object-based, use APIs rather than traditional storage protocols, and include per-object metadata. In fact, it is fairly straightforward to integrate today's cloud storage systems into applications developed to leverage CAS. But cloud storage goes further in terms of application integration and programmability (take a look at the Nirvanix API, for example). Vendors have added many features, from replication to indexing to media transcoding, each of which can be called by applications through custom APIs. Cloud storage also leverages the openness of the Internet and modern programming concepts, incorporating the Internet Protocol (IP), HTTP, SSL, REST, and SOAP. This is not to say that cloud storage can only be used by specialized applications, however. Most cloud systems include basic web browser interfaces. More interestingly, many interface solutions have been developed to bridge traditional storage protocols to the cloud. One major contributor to the success of Amazon's S3 storage offering was Jungle Disk, a consumer-oriented application that allows users to automatically back up their files to the service. Nirvanix developed CloudNAS for enterprise users, which presents cloud storage service as a Linux filesystem or Windows drive. And EMC and Emulex recently revealed that they are working on a bridge between block-based SANs and cloud storage. Although it can be leveraged by existing applications, often at lower cost, the real benefit from cloud storage comes when applications take advantage of its compelling distribution, collaboration, and programmability capabilities. The entire storage industry is moving toward greater levels of application awareness and integration. LUNs (fake disk drives) served up by SAN arrays are being hidden behind shared file systems in the server virtualization space. NAS is also being updated for greater integration with applications. This is a necessary step to bring about a real storage revolution that will see a transition from bulk management of capacity to granular management of data to integrated use of information. That's why cloud storage is different, and why cloud storage matters! Posted Jun 19 2009, 07:01 PM by sfoskett with 4 comment(s) Filed under: HDS, Amazon, EMC, virtualization, archiving, backup, VTL, CloudNAS, private cloud, Jungle Disk, SAN, Nirvanix, Dell, S3, Caringo, DAS, Atmos, 3PAR, secondary storage, primary storage, NAS, tiered storage, CAS, Seagage

• Is A Private Cloud Worthwhile?

  Much discussion in the cloud computing world has focused on a simple question: Is a private cloud infrastructure worthy of the name? It's been posed in many ways, with some going so far as claiming that there is no such thing as a private cloud. Although discussions like these are all too common in many areas, the question really amounts to little more than counting angels dancing on pin heads. The key issue is whether private cloud-style infrastructure can deliver real benefits like public clouds can. First, let's set out some definitions: The draft NIST definition, perhaps the best we have at this point, states that "Cloud computing is a pay-per-use model for enabling available, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, services) that can be rapidly provisioned and released with minimal management effort or service provider interaction." McKinsey's controversial discussion document tells us that there are four key benefits of the cloud: Faster time to market, creation of new value drivers, easier scale-out, and lower upfront IT costs. These benefits were independently demonstrated by Roman Stanek of Good Data, and serve as a roadmap for cloud success. Defining what is and isn't a private cloud can be difficult, but the word "private" gives some clues. Simply put, a private cloud is closed off, existing solely for a single organization's use. One of the key value propositions for cloud computing is the transfer of expense from the capital (CAPEX) to the operational (OPEX) column. In other words, using a service (like, ahem, the Nirvanix SDN) means there is no hardware or software to buy, built, and maintain. A simple recurring cost replaces an entire set of in-house processes and equipment. This is impossible if one builds their own "private" cloud. But does the loss of this important benefit really mean that public clouds are worthless? Not at all! Private clouds can still deliver some of the other benefits of cloud computing, especially for the largest organizations. Private and hybrid clouds can also serve as a gateway, allowing enterprise IT to become familiar and comfortable with cloud computing paradigms in a controlled environment. Some private cloud applications may eventually be migrated to shared or public cloud infrastructure, but others might remain in-house permanently. Consider the case of private, shared, and public air travel. Many of the biggest companies maintain their own stable of corporate aircraft. This might seem foolish to the average person, or even the travel departments of medium-sized businesses, but the substantial expense might be offset by the convenience or increased productivity of private aviation. Cloud computing is similar: The average individual or organization will probably derive maximum benefit from sharing a public cloud infrastructure, but this should not preclude certain special cases where a private cloud will be called for. This is not a zero-sum game. The concept of cloud infrastructure is so strong that we should all stop worrying if this or that definition stands up. The future is coming, and it includes transformed and virtualized private infrastructure as well as public cloud providers. Posted Jun 04 2009, 09:13 PM by sfoskett with 3 comment(s) Filed under: capex, opex, public cloud, private cloud, Christofer Hoff, Sam Johnston, Ruven Cohen, McKinsey, Chuck Hollis, NIST, Roman Stanek

• Is Cloud Storage The Right Tool For The Job?

  The current economic climate is pushing many to look for current-year budget savings. This is one of the major reasons that cloud computing in general, and cloud storage specifically, has become so attractive. Moving data into the cloud doesn't just get it out of the data center, it converts a capital expense (storage equipment) into an operational one (monthly fees). But cloud storage isn't right for every application: One must profile applications and consider where cloud storage or computing services makes sense. The folks over at Cleversafe recently proposed five questions to ask before jumping into cloud storage: "What Type of Data do You Need to Store? How Much Control Do You Want to Have Over your Data? What type of Cloud Storage Best Suites your Business? How Does Your Prospective Cloud Vendor Geo-disperse? When it Comes to Storage, is it Really All About the Benjamins?" These are great questions, but application profiling goes a lot further. One must consider many aspects, as the Fountainhead blog suggests for cloud applications. If we generically consider which elements we should assess when profiling applications for cloud storage, we come up with four basic attributes and one overriding factor: Cost! Performance - Will the cloud storage service provide the level of throughput we need? Consider the network latency as well as any latency associated with interacting with a cloud storage API! Security - Consider all three elements of the security triad: Confidentiality, Integrity, and Availability. Compliance - How will the service meet your disaster recovery needs? Will it protect data operationally? What about regulatory needs like archiving? Support - How quickly must the service provision and tear down service? How responsive is the support organization? We will delve into these questions in detail in future posts. Stay tuned! Posted Jun 03 2009, 11:20 PM by sfoskett Filed under: cloud computing, cost savings, cost avoidance, performance, cloud storage, profiling, security, Cleversafe, compliance, support, questions