My wife and I are home-schooling my 8th grade daughter. As part of her science class this week, she had to do an experiment to see if objects fall at the same rate when dropped from the same height. The experiment started with dropping a piece of paper, cardstock, and a book (all of the same size), determining which hit the floor first. With the book landing on the floor first, my daughter concluded that things fall at different rates based on their weight. But then she crumpled up the piece of paper and dropped it. She was surprised that the book and crumpled paper landed at the same time. If forced her to challenge her initial conclusion that objects fall at different speeds based on their weight. She realized that objects fall at the same rate if you remove the affects of air resistance.
So what does this little science experiment have to do with storage? Over the last several weeks there has been a couple of discussions, one on capacity efficiency and the other on performance.
The capacity efficiency discussion was initiated by EMC. They calculated theoretical capacity efficiency of several disk arrays, including the StorageWorks EVA, which initially resulted in very favorable results for the midrange EMC arrays. It was far from science and I'd say heavy on artful marketing. I won't try to catch you up on all of those discussions but you could read a couple of posts on this site to get "caught up". Those posts are:
HP has suggested a couple of times that we'd be more than happy to participate in a third party evaluation of capacity efficiency - a real storage capacity science experiment. It seems to me what customers need is real data and not one vendors' biased view that does nothing but add to our customers' confusion. To date, EMC has not responded to our challenge - they didn't even acknowledge our offer. I do appreciate that Chuck Hollis did admit on his September 12 blog entry that his analysis was wrong. Here's what he said:
I think we got things wrong on the HP EVA. We configured 7 disk groups, HP recommended 2. The rules of the game were clear: go with what the vendor recommended. So we were wrong in that regard. Even with the changes, I think EMC still has a usable capacity advantage, but it's not as pronounced.
I appreciate Chuck's honesty here but as of now, there's nothing conclusive that either array has any advantage. I don't appreciate that he still claims that EMC has an advantage - he has no proof and no substantiation. I believe the best path forward would be for EMC to agree to a third party assessment - a real storage science project - instead of claiming some victory that doesn't exist.
Another storage-focused science experiment was discussed last week too. In this case, it was around storage performance and it did have the makings of a real science experiment. HP announced the results of our SPC-2 benchmark. This testing is defined and standardized by the Storage Performance Council (SPC). The SPC is a vendor-neutral standards body focused on the storage industry. You can read more about them on the SPC website. Every major storage vendor and even many of the smaller ones participate in the SPC. Guess who refuses to? Yup, EMC. Here's what Chuck wrote in his September 12th blog:
Not a month or two seems to go by without some vendor running an SPC test, bragging about the results, and challenging EMC to a showdown. And, just as predictably, we go look at what they've done, shake our heads, and further resolve never to get involved in any of this nonsense.
This all has me shaking my head. EMC refuses to participate in meaningful, third party validated testing around capacity efficiency and performance but is more than happy to artfully market their "leadership" in these areas based on who knows what. Am I biased - absolutely I am. But I also totally get the fact that marketing fluff and spin doesn't benefit our customers when trying to make decisions. I hope in the future these topics are covered by EMC with more of a scientific rigor instead of marketing talking points and spin.
As customers, you have to challenge what vendors are saying - make sure that their claims are backed by substantiation. Be careful when you hear about experiments (remember my daughter's science experiment) that ignore facts and give wrong conclusions. HP's "Standards of Business Conduct" require that what we say can be substantiated. Does that mean that we are always perfect - no, of course we aren't. But I think it means that customers can trust what HP says... so challenge us.