Around the Storage Block Blog

HP and Dell comparison on Exchange 2010 Migration: Why you need to understand what testing really took place

by on

By Ian Selway, Worldwide HP Storage Solutions Manager

As the Microsoft solutions marketing manager involved with the recent HP and Microsoft announcement, I receive a lot of information about what our competitors are saying around their offerings for Microsoft applications. What staggered me about a recent piece of Dell customer literature was the claimed cost-benefit conclusions drawn by the third-party testing organization selected by Dell for a Microsoft Exchange 2010 migration. It compared a four-year old HP ProLiant server using an HP StorageWorks MSA 1000 against new Dell servers using internal SATA drives.

Depending on your thoughts about vendor-sponsored "independent" third-party testing (something I'll admit that HP also engages in), you may already have an opinion as to the validity of such comparisons. What amazed me about this particular piece was the comparison itself-coupled with conclusions drawn from the so-called evaluation of the two offerings. In most reviews, testers attempt to assess the merits of similar systems. In this case, the comparison was of a four-year old HP architecture versus a new Dell configuration. This is one of the most obvious apples vs. oranges testing I've ever seen put forward for serious consideration.

So let me point out just a few of the dubious conclusions drawn:

Better messaging performance: The third-party organization choose the testing platforms. Yet even with a Dell server with the latest 64bit OS, an 8-processor core using hyper threading (16 logical processors) and 12GB memory, the Dell machine was only capable of a .12 message-per-second improvement over an HP solution that was more than the four years old and was only using a 32bit OS with 2-processor cores and 4GB memory.

Operational cost: I know HP has excellent management tools but it was nice of Dell to acknowledge that by implementing the Dell Exchange solution, organizations could expect to pay up to $50K extra in Exchange administration costs compared to the HP offering.

End-user productivity: Dell's proposition-and a huge proportion of their "estimated" cost savings-is that end users would benefit from much larger mailboxes by not having to spend up to three minutes per day managing their 60MB mailbox. Now I don't know about you, but in my experience, I can't see any difference between what I do now with a 250MB mailbox, and what I did three or four years ago with a 50MB mailbox. I still file messages. I still manage my inbox. For all that I love my messaging system and the productivity gains provided, it is still a tool. Like paper systems of old, it requires process management and rigor. Now with a 600MB mailbox, I may not have to purge messages as regularly, but I'd still have to do the similar routine maintenance and that will take time out of my day.

Power savings: At a time when it's good to show your green credentials, companies are trying to show how their new hardware saves you money in terms of power draw. In Dell's case (just to make sure they showed they had an advantage), not only do they add the servers and power supplies in the MSA1000 (which is fine as you need all this HP hardware but hardly a like-to-like comparison), they configure it so that power management is set to always-on for the HP servers while their own hardware was optimized and balanced for power consumption. I would have thought just comparing a new, hopefully better designed and thus more power efficient server to four-year old system would be enough. I'm sure if you used the latest HP hardware with our sea of sensors, improved power supplies and fans, we'd show significant power savings as well.

As I stated earlier, if you're going to do a compare, wouldn't you at least want to make it reasonably hard for those reading the analysis to see how you've rigged the testing?

In conclusion, Dell would have been better served by paying their independent testers to actually compare current systems from HP and Dell and then determine the cost-benefit analysis of deploying on either vendors hardware. I know why they didn't take this approach. When you look at the analysis, the cost of managing Microsoft Exchange on HP infrastructure vs. Dell is significantly cheaper ($25K vs. $75K). The third-party testers claim that end users would spend significantly more time managing mail in Exchange 2003 than they would using Exchange 2010.

The conclusions drawn have nothing to do with either vendor's hardware. It's really about the new features, storage architecture choices and management capabilities in Microsoft Exchange 2010. Where I believe HP does have an edge is in the choice of storage for Microsoft Exchange 2010, combined with the more than 25 years of tight engineering as part of the ongoing HP and Microsoft partnership. So come on Dell...why not do a real comparison-one that customers can have confidence in, and one that compares apples to apples?

Labels: