“Small Data”

[#1]

One of the great things about technology is that when times get tough, it sometimes results in innovations that people didn't think were possible. We're seeing this today in the automobile industry, where the best companies in the business are exceeding expectations on fuel efficiency. It used to be the case that if you wanted fuel efficiency you had to make huge compromises in performance and capability. This year there are a number of new cars coming out that actually have a lot of performance, are fun to drive, and offer tremendous fuel efficiency. In the last quarter there was a significant shift in volume toward smaller cars. Detroit is expanding capacity for small car production. This change is purely a function of the creativity of the best technologists in these auto companies who are responding to the dual challenges of environmental impact and $4/gallon gas.

In enterprise storage we have similar challenges.  We are experiencing a tremendously high rate of change.  Capacity requirements are growing at an unprecedented rate, and there are more new applications and use cases than ever before. At the same time, we've experienced constraints in availability of certain types of disk drives and increasing costs. The good news is that, as in the auto industry, the best companies in storage are innovating rapidly in the area of efficiency.  We now have customers that are using new HP efficiency technologies to not only mitigate the costs of increasing capacity requirements, but to actually decrease the amount of money they are spending on disk drives.

One of the interesting areas of technology innovation is what I call “small data”.  At HP we have made a significant investment in finding ways to reduce the amount of data that is stored, and fit as much data as possible on storage media. One of the key techniques here is to “atomize” the data and break it down into the smallest possible chunks as we store it to disk.  Our HP 3PAR systems use a sophisticated logical engine to place the data on the most appropriate available space, based on the level of service that is required. 3PAR uses fine-grained allocation unit size of just 16 KB, as compared to 1 MB - 17 GB with other systems.  This prevents small writes from consuming megabytes or even gigabytes of capacity or from filling volumes with empty space.  In this way we compact much more information onto the storage media than traditional storage technologies. The result is that many customers that were previously operating at 20% or 30% storage capacity utilization and wasting money, are now running at 70%-80%. We had a customer recently where we were able to reduce their costs by over $10 million in a 12 month period by turning on software capabilities in HP 3PAR storage systems they already had.

Another example of “small data” is our HP StoreOnce family of backup systems.  These systems use a sophisticated set of data deduplication algorithms that were developed by HP Labs.  Deduplication algorithms are measured by their ability to deliver high data reduction ratios. StoreOnce employs a chunk size of 4k—the smallest in the industry. By comparison, older deduplication technologies use chunk sizes of 8k, 16k, or 32k. Smaller data chunk sizes lead to a higher percentage of data “matches” and higher deduplication ratios. This is because it is easier to match a smaller data pattern than a larger one, and to do so more quickly. Even small improvements in the match ratio deliver multiplier effects for the amount duplicate information that can be identified.  For example, a 95% match ratio delivers a 20x deduplication ratio. 1/(1–.95) = 20. If you can achieve a 3 percent higher match to 98%, you get a 50x deduplication ratio. 1/(1–.98) = 50.

These are but two examples of advanced “small data” efficiency technologies that HP has brought to our customers during the last two years.  Ironically, one of the keys to managing data growth is to use technologies that can help you “get small”, buy fewer disk drives, and save money.

For more information:

HP 3PAR Storage

HP StoreOnce Backup Systems

[#2]

Hello.

Your post is both seems to reflect some of the ternds we are also experiencing in our industry.

By coincidence we have just had new HP equipment installed in our office, which is working well by the way and we have wittiness a trend in customer applications in the rise in small vehicle usage.

I must admit I did not fully understand the whole article, a little too technical for me but the trend in smaller and more would seem to taking hold.

As you point out, with gas at $4 it's a real mover in the market place.

Thanks for an interesting article.

Yours,

Malcolm.