Ever have a feeling that something's holding you back? With ProLiant server blades and Windows Server 2008 R2, you can find out if it's true.
Mike Manos responded to my post about power capping being
ready for prime time with a very well thought out and argued post that really
looks at this from a datacenter manager's perspective, rather than just my
technology focused perspective.
I'm going to try and summarize some of the key issues that
he brings up and try to respond as best I can.
This one spans a number points that Mike brings up, but I think the key thing here is that you
must have a critical mass of devices in the datacenter that support power
capping otherwise there is no compelling value.
I don't believe it is necessary, however, to have 100% of devices in the
datacenter that support power capping. There
are two reasons why:
In most Enterprise datacenters the vast majority
of the power for the IT load is going to the servers. I've seen numbers around 66% servers, 22%
storage and 12% networking. This is a
limited sample so if you have other numbers let me know I would be interested.
Most of the power variation comes from the
server load. A server at full load can use 2x - 3x the power of a server at
idle. Network switch load variation is
minimal based on some quick Web research (see Extreme Networks power consumption test or Miercom power consumption testing). Storage power consumption variation also seems to
fairly light at no more than 30% more than idle. See Power Provisioning for a
Warehouse-sized Computer by Google
So if our Datacenter manager, Howard, can power cap the
servers then he's got control of the largest and most variable chunk of IT
power. Would he like to have control of
everything, absolutely yes, but being able to control the servers is more than
half of the problem.
Been there done that,
got the T-Shirt
The other thing that we get told by the many Howards that
are out there is that they're stuck.
They've been round and round the loop Mike describes and they've hit the
wall. They don't dare decrease the
budgeted power per server any more as they have to allow for the fact the
servers could spike up in load, and if that blows a breaker taking down a rack
then all hell is going to break lose.
With a server power cap in place Howard can safely drop the budgeted
power per server and fit more into his existing datacenter. Will this cost him, sure, both time to
install and configure and money for the licenses to enable the feature. But I
guarantee you that when you compare this to cost of new datacenter facilities
or leasing space in another DC this will be trivial.
I agree most datacenters are in fact heterogeneous at the
server level either; they will have a mix of server generations and
manufacturers. This again comes down to
critical mass, so what we did was enable this feature on the two of the best
selling servers of the previous generation, DL360 G5 and DL380 G5 and pretty
much all of the BladeSystem blades to help create that critical mass of servers
that are already out there, then add on with the new G6 servers. We would of course love for everyone with
other manufacturer's product to upgrade immediately to HP G6 ProLiant Servers
and Blades, but it's probably not going to happen. This will delay the point at which power
capping can be enabled and for those customers that use other vendors systems
they may not be able to enable power capping until those vendors support it.
Power Cap Management
There's a bunch of issues around power cap management that definitely
do need to get sorted out. The HP
products do come from an IT perspective and they are not the same tools that facilities
managers typically use. Clearly there
needs to be some kind convergence between these two toolsets even if it's just
the ability to transfer data between them.
Wouldn't it be great if something like the Systems Insight Manager/Insight
Power Manager combination that collects power and server data could feed into something
like say Aperture (http://www.aperture.com/)
then you'd have the same information in both sets of tools.
The other question that we have had from customers is who
owns and therefore can change the power cap on the server, the
facility/datacenter team or IT Server Admin team. This is more of a political question than
anything else, and I don't have a simple answer, but if you are really using
power caps to their full potential changing the power cap on a server is
something that both teams will need to be involved in.
I would like to know what are the other barriers you see to
implementing power capping - let me know in the comments and be assured that
your feedback is going into the development teams.
Just to make Mike happy I thought I'd let you know that we
do have SNMP access to the enclosure power consumption.
If you collect all six SNMP MIB power supply current output
power values and add them together, you will have calculated the Enclosure
In the CPQRACK.MIB file, which you can get from here http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?swItem=MTX-a7f532d82b3847188d6a7fc60b&lang=en&cc=us&mode=3&
There are some values
cpqRackPowerSupplyCurPwrOutput which is MIB item:
enterprises.184.108.40.206.220.127.116.11.1 through enterprises.18.104.22.168.22.214.171.124.6
gives you the Input Power of each Power Supply, I know the MIB name says output
but it's actually input - sum these together then you have the Enclosure Input Power.
Power supplies placed in standby for Dynamic Power Savings
will be reporting 0 watts.
And for enclosure ambient temp - read: