World Library  
Flag as Inappropriate
Email this Article

Storage efficiency

Article Id: WHEBN0022142103
Reproduction Date:

Title: Storage efficiency  
Author: World Heritage Encyclopedia
Language: English
Subject: Apache Thrift, File system
Collection:
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Storage efficiency

Storage efficiency is the ability to store and manage data that consumes the least amount of space with little to no impact on performance; resulting in a lower total operational cost. Efficiency addresses the real-world demands of managing costs, reducing complexity and limiting risk. The Storage Industry Networking Association (SNIA) defines storage efficiency in the SNIA Dictionary as follows:

\text{storage efficiency} = \frac{\text{effective capacity} + \text{free capacity}}{\text{raw capacity}}.

The efficiency of an empty enterprise level system is commonly in the 40%–70% range, depending on what combination of RAID, mirroring and other data protection technologies are deployed, and may be even lower for highly redundant remotely mirrored systems. As data is stored on the system, technologies such as deduplication and compression may store data at a greater than 1-to-1 data size-to-space consumed ratio, and efficiency rises, often to over 100% for primary data, and thousands of percent for backup data.

Technologies

Different technologies exist at different and sometimes multiple levels:

Snapshot technology—known formally as "delta snapshot technology"—gives the ability to use the same dataset multiple times for multiple reasons, while storing only the changes between each dataset. Some storage vendors integrate their snapshot capabilities at the operating system and/or application level, enabling access to the data the snapshots are holding at the system and/or application management layers. Terminology around snapshots and "clones" is currently confusing, and care must be taken when evaluating vendor claims. In particular, some vendors call full point-in-time copies "snapshots" or "clones", while others use the same terms to refer to shared-block "delta" snapshots or clones. And some implementations can only do read-only snapshots, while others are able to provide writable ones as well.

Data deduplication technology can be used to very efficiently track and remove duplicate blocks of data inside a storage unit. There are a multitude of implementations, each with their separate advantages and disadvantages. Deduplication is most efficient at the shared storage layer, however, implementations in software and even databases exist. The most suitable candidates for deduplication are backup and platform virtualization, because both applications typically produce or use a lot of almost identical copies. However, some vendors are now offering in-place deduplication, which deduplicates primary storage.

Thin provisioning technology is a technique to prevent under-utilization by sharing the allocated, but not yet utilized capacity. A good example is Gmail, where every Gmail account has a large amount of allocated capacity. Because most Gmail users only use a fraction of the allocated capacity, this "free space" is "shared" among all Gmail users.

Major advantages

Actively increasing storage efficiency using these techniques has the following advantages:

Backup and restore. Using snapshots, time used for both backup and restore RTO can be minimized. This can greatly reduce cost, and reduce hours of downtime to seconds of downtime. Snapshots also allow for better RPO values.

Reducing floorspace. When less storage is required to store a given amount of data, less data center floorspace is required.

Reducing energy use. When fewer spindles are required to store a given amount of data, less power is required.

Provisioning efficiency. Writable delta snapshot technology allows for very fast provisioning of writable data copies. This reduces waiting time in processes that require that data. Examples are data mining, test data, etc. Snapshot integration at the OS and/or application level also leads to faster provisioning, because system and/or application managers are able to manage their own snapshots without having to wait for storage managers and/or provisioning procedures.

Major commercial players

All major vendors are implementing one or more of these technologies, because storage efficiency is becoming more and more popular. Customers are facing storage requirements that are growing exponentially and a strong demand for cost-cutting. The major vendors are NetApp, EMC, HDS, IBM and HP.

This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 


Copyright © World Library Foundation. All rights reserved. eBooks from Project Gutenberg are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.