BDPrime writes "We've all been hearing ad nauseum about power and cooling issues in the data center. Now the EPA has issued a final report to Congress detailing the problem and what might be done to fix it. Most likely what will happen is the EPA will add servers and data centers into its Energy Star program. If you don't feel like reading the entire 133-page report, the 14-page executive summary is a little easier to get through."
These forecasts indicate that unless energy efficiency is improved beyond current trends, the federal government's electricity cost for servers and data centers could be nearly $740 million annually by 2011, with a peak load of approximately 1.2 GW.
That sounds like a big number, and is for most of us, but not for the Federal government. About 29 cents more in taxes off each paycheck (assuming 100 M taxpayers, and paychecks every 2 weeks).
There are much bigger fish to fry.
Also, there is only so much one can cut the energy use, and thus that cost down, and still get the business of the government done. And the improvements in efficiency will require hardware, software, and personnel which have their own costs. Eventually you will hit a point where there is no longer a return on investment to make it worthwhile.
Notice that the report was issued by the EPA, and what seems like a trifling amount of money for the federal gov't could represent a very significant impact to the environment. On page 8 of the summary it projects the CO2 emissions that could be avoided under the different scenarios.
Also, there is only so much one can cut the energy use, and thus that cost down, and still get the business of the government done. And the improvements in efficiency will require hardware, software, and personnel which have their own costs. Eventually you will hit a point where there is no longer a return on investment to make it worthwhile.
The same thing was said for many other things over the years; lighting pops to mind. Offices used to consume about 3 watts per square foot of office area in the 70's.
Maybe in a sentence.. "Individual Server power usage is embarrassing and it should be more efficient"... So what happens now?? Now we wait for a congressional committee meeting broadcasted on c-span where politicians can grand stand and talk about how it needs to change... Fortunately most politicians will consider this typical as a "black box", so they will not touch on technical details, but rather just complain.. Maybe talk about a special "colo server" tax.... Maybe throw in some global warming comments.
Having any Govt investigate efficiency is about as practical as the Madonna Commission On Chastity and Modesty.
Computers are doing just fine at reducing their power consumption by many percent a year without the govt's "help".
n 2006, U.S. data centers consumed an estimated 61 billion kilowatt-hours (kWh) of energy, which accounted for about 1.5% of the total electricity consumed in the U.S. that year.
1.5% of the total electricity used in the US per year is a huge number. It's like when politicians talk about something really expensive and they say "oh, it's only 1% of our GDP" to make it sound not so bad, except to people who know just how enormous the GDP of this country is.
More importantly, this could probably be reduced considerably without major disruptions or reduction in quality of service by just embracing higher efficiency components in our datacenter equipment (especially servers).
It's an estimated 11,000,000 servers in everything from 2 server closets to thousand server enterprise centers. These 11 million systems consume more power than all the TV sets in the US combined, and there are more TV sets in the US today than people.
Or lets do it this way. Hoover Dam at peak output produces 2 Gigawatts of power per hour. 11 million servers consume 61 billion KW hours annually. It takes Hoover Dam 30,000 hours (about 3.5 years) to produce that much power. So you need four Hoover Dams just to power all the data centers in the US.
Actually, dams are serving as magnets for data center development, since hydro power is cheaper than other sources and provides the public relations advantage of being "greener" than coal or nuke power. That's why more than 2 million square feet of data center space [datacenterknowledge.com] is being planned in and around Quincy, Washington, a farm town of 5,000. Meanwhile, in northern NY state, HSBC is locating a $1 billion data center project [datacenterknowledge.com] in Cambria (another farm town of 5,000), where it will use hydro power from the Niagara r
Or lets do it this way. Hoover Dam at peak output produces 2 Gigawatts of power per hour.
What you meant to say is "Hoover Dam at peak output produces 2 Gigawatts." What does make more sense is saying 48 million KWH per day or a bit over 17 billion KWH per year - assuming that there is enough water behind the dam to allow for continuous peak output, which is certainly not the case this year.
Ya know, I always wondered why most places weren't more efficient about the cooling of their datacenters... particularly in the winter. Like it'll be 20 degrees F outside and they're STILL running A/C for the computers. WTF? Just vent a small amount of the outside air into the datacenter and you're done. Or better yet, just blow in the air from the offices and send them warm, data center heated air.
Another question, why do we vent the exhaust from our refrigerators into the house during the summer? Just seems like there's a lot you could do to save energy just by moving what would outerwise be waste heat to places where it can either be used or at least not cause a larger cooling problem.
Like it'll be 20 degrees F outside and they're STILL running A/C for the computers.
Climate controlled. There's this element among building planners that think any outside air is bad(TM). That's why, even in small buildings where you don't have to worry about pressure differentials blowing windows out like you do in skyscrapers, you can't open a frick'n window in the Fall or Spring when the air smells wonderful and there's this perfect chill in the air the just stimulates the brain.
I'm drenched in sweat here in Hotlanta (it's 82F and 66% humidity and climbing to 94) and I really miss New England's Spring and Fall.
The concept of "free cooling" is gaining significant momentum in the data center space. I don't have any free or public information, but rest assured that leveraging winter air and other technique are being looked at very hard. Of course, this is not altruism or "green" thinking. It's our old friend financial greed. Reduction in capital expenditures for chiller plants and reduction in utility bills.
Humidity, pollution and other things as well as physical access must be controlled.
Pollution is easily taken care of with a filter. Controlling physical access is trivial. Humidity may be a bit more involved, but then again, you're heating the incoming air which reduces its relative humidity. Condensation isn't likely. If it does turn out to be a problem, use a heat exchanger and preheat the incoming air using the exhaust air.
But it is not terribly practicle to plug the plenum passages once a year.
I've long been dumbfounded by the way datacenters charge. They seemingly all charge a hell of a lot for physical space, and then almost completely ignore power requirements. This seems incredibly strange, since datacenter operating costs are pretty much tied directly to power consumption (monthly electricity fees, UPSes, electrical generators, cooling, etc.), and only incidentally to physical space.
Further, the cost to handle each extra watt is multiplied thanks to cooling, power back-up, wiring, etc., while increasing the physical size of the building, constructing more datacenters, etc. is just a flat (linear) cost, and mostly just a one-time expenditure at that.
This strange arrangement is what has led us here. It's not the natural evolution of technology to cram as much power consumption into as tiny a box as possible. It's an artificial need, created by the idiotic distribution of fees common to datacenters.
If a few large datacenters declared their fees as a small $$$ value for each unit of space, and additionally a few dollars, per watt of power consumption, you'd see the problem naturally fix itself, through normal economic forces. As soon as watts are the defining factor, companies won't pay more for a cramped 1U server rather than an (inexpensive) 2U or 3U server. You will also see companies happy to pay more for lower-powered server hardware, as having them directly bear the energy cost will make buying efficient servers a significant savings to them.
People good with numbers will usually take the usable floor space of a data center and put a watts per square foot estimate with it based on average or projected power consumption. If you say every square foot costs an average of $5 in power usage, you can figure that in with maintainance per square foot, cooling per square foot, etc, etc. Combine these all into a neat little sum on someone's bill.
Try to think of data center 'floor space' as the main stage. Everything is built around maintaining and supply
People good with numbers will usually take the usable floor space of a data center and put a watts per square foot estimate with it based on average or projected power consumption.
Of course the (average) price of electricity is figured into it. That is the PROBLEM.
It is a (self-perpetuating) prisoner's dilemma. The more power consumption you can squeeze into the smallest space, the better of a deal you get. Since it's all averaged out, those using more power than average are getting subsidized by those w
Very few, if any providers charge solely by square footage anymore. Like shipping packages where the fees are combination of volume, weight, distance, etc. Data center pricing is typically priced based on a combination of space, power, power density, circuits, contract length, continguous space, etc.
If a few large datacenters declared their fees as a small $$ value for each unit of space, and additionally a few dollars, per watt of power consumption, you'd see the problem naturally fix itself, through normal economic forces
How on earth do you track individual power consumption? Putting a meter on each system is hardly practical. I suppose you get away with one on each rack, but many customers (the vast majority in the one data center I worked in) don't rent whole racks.
This is already happening to a functional point. Navisite, the host we use here at this company, charges by the square foot, however you only get so many watts per square foot. We have (2) 19" racks about half full of hardware with a total physical footprint of under 9 square feet; we could even get it under 5. However due to the power density we have 100 square feet of space that we rent. Because we use hyper dense blade servers for the management efficiency we fill a "racks" space of power with aproxa
It probably would cost to much to bother reporting on...
Because when you run a multi-million dollar data center, you clearly can't afford install a few-hundred dollar device in each customer's rack especially if it's a major part of how you bill your customer.
Look, the power companies do exactly what the parent poster suggests. Imagine if power companies charged a flat rate each month based on the square footage of your house. There would no incentive (unless your a save-the-planet hippie type which isn't a bad thing) to turn up the setting on the air conditioner (or turn it off all together), keep incandescent lights running 24/7 along with the giant plasma TV. This is essentially how data centers operate today. There is no motivation to have energy efficient servers unless you're the one that owns the data center and pays the power bill. Today the best a data center owner can do is invest in more efficient cooling systems and that's about it.
I just grabbed the executive summary version, and didn't see any mention of virtualization..
To me, this seems like one of the more important aspects of power efficiency. Individual server efficiency is important, but the gains from higher utilization could be even more significant. Adding another core to a hypervisor will always be more efficient than adding a new system (CPU, Power Supply, disks, video, etc..). The energy efficient hardware can also be applied to the hypervisor hosts. Build efficient servers, and use as few of them as practical.
Many data centers are already greatly decreasing their server count using virtualization. This should be part of any data center energy efficiency discussion.
Virtualization is great for some things, but security concerns are substantial - PCI compliance generally means that all of the guests on a host have to be at the same security level. Also, some of the virtual environments don't handle IPv6 properly (and a few other things). These aren't showstoppers, but they can reduce some of the benefit.
Also, in most current virtualization environments, performance under heavy load on the guest has shown to suffer from 10-40% (depending on which virtualization product you're using). Not quite ready for prime time yet.
Table ES-1 in the executive summary suggests server consolidation at various levels (moderate, aggressive, etc.). Server consolidation can be done in a number of ways, with virtualization being one of the most effective and popular.
The report addresses virtualization only indirectly when it refers to electric utilities offering incentive programs. PG&E offers financial incentives to encourage the use of virtualization in data center consolidations [datacenterknowledge.com], with qualifying customers able to earn a rebate of up to $4 million per project site. Other utilities are looking at adapting similar incentives based on virtualization.
I'm not sure EPA is the right party to be advocating virtualization. The EnergyStar ratings and utility-level programs
With a lot of these massive datacenters residing in sunny california you tihnk they could offset a large chunk of their power needs with solar panels covering the roofs Like the FedEx hub in Oakland.
Yes, but how much less heat? DC rectifiers waste heat too. 3-phase power supplies on blade chassis are considerably more efficient than typical 1-phase pizza box power supplies. The blade system has an efficient entry cost an order of magnitude more than the pizza box, and 3 orders of magnitude less than the DC data center, but is closer to the DC data center in AC conversion efficiency. This makes it a very good compromise for the vast majority of enterprises that do not buy their data centers by the a
OK, so I am a desktop/notebook guy. So this stuff may already exist for servers - but:
1) With multiple front-end servers behind NLB, make the NLB smart enough to put some servers to sleep when their processing isn't needed and wake-on-lan those servers (or the equivalent) when they are needed again?
2) Do servers do "speedstep" like desktops/notebooks where the processors and other components go to lower power level modes when they are not being fully utilized? If not, they should enable that.
AMD processors "speedstep" (it's why we purchased AMD boxes for our hosting operation). We've saved a huge amount on power (our datacenter does metered power, not flat rate power) due to this.
Most servers are also overpowered for the workload they perform. Just getting people to stop running 3GHz processors when a 1Ghz Sempron would do, would be a nice start.
I have found that stock switching power supplies as found in common computers are slightly more efficient when powered with 240 volts rather than 120 volts. Some more so and some less so. And virtually all of them can be changed over to 240 volts (having the correct 2-pole switching).
And by using 240 volts instead of 120 volts, you can run twice as many computers on the same power loss in the building wiring (same current, same size wire, same power loss due to heat, serving twice the load).
Direct DC fed power systems may or may not provide realistic savings. DC introduces new electrical safety challenges and costs (electrical arcs inside switches, circuit breakers, and fuses, cannot be cut off by AC's zero voltage crossing that DC does not have). This requires lower voltages for equivalent interruption safety. But if power supplies end up losing less power than the building wiring at the higher current, then DC may be the better choice.
We will need more in-depth study to determine if DC will save power or not at a given installation (it may at some and not at others). But for most installations, going from 120 volts up to 208 or 240 volts (depending in which is available) is as simple as rewiring the system (using 2-pole breakers... requiring double size power panels) and verifying the computer power supplies are ready for the higher voltage.
208 volts is the likely line-to-line voltage in data centers powered by 3-phase (208Y/120) power in North America. Future data centers could be designed for a 416Y/240 volt power system which can also be used to power fluorescent lighting.
Summery (Score:3, Funny)
Still too long. Can anyone reduce it to a single phrase or word? Thanks in advance
Great scott! (Score:5, Interesting)
Parent
Re:Great scott! (Score:5, Funny)
Parent
Re:Great scott! (Score:4, Insightful)
($177M/day for Iraq http://www.usatoday.com/news/politicselections/na
That sounds like a big number, and is for most of us, but not for the Federal government. About 29 cents more in taxes off each paycheck (assuming 100 M taxpayers, and paychecks every 2 weeks).
There are much bigger fish to fry.
Also, there is only so much one can cut the energy use, and thus that cost down, and still get the business of the government done. And the improvements in efficiency will require hardware, software, and personnel which have their own costs. Eventually you will hit a point where there is no longer a return on investment to make it worthwhile.
Parent
Re: (Score:2)
Re: (Score:3, Insightful)
The same thing was said for many other things over the years; lighting pops to mind. Offices used to consume about 3 watts per square foot of office area in the 70's.
Our government, now powered by lightning (Score:2)
The only power source capable of generating 1.21 gigawatts of electricity is a bolt of lightning.
(Just reinforcing the reference. heh)
Re: (Score:2)
So what happens now?? Now we wait for a congressional committee meeting broadcasted on c-span where politicians can grand stand and talk about how it needs to change... Fortunately most politicians will consider this typical as a "black box", so they will not touch on technical details, but rather just complain.. Maybe talk about a special "colo server" tax.... Maybe throw in some global warming comments.
Grampa Simpson: (Score:2, Funny)
Re:Grampa Simpson: (Score:4, Funny)
Plead your case.
Parent
Re: (Score:3, Funny)
*runs from the hounds*
Mandatory Madonna reference (Score:3, Funny)
Re:Mandatory Madonna reference (Score:5, Funny)
Which Madonna?
Parent
wow (Score:3, Informative)
Is that it? Seems like small potatoes to me.
Re:wow (Score:5, Interesting)
More importantly, this could probably be reduced considerably without major disruptions or reduction in quality of service by just embracing higher efficiency components in our datacenter equipment (especially servers).
Parent
Re:wow (Score:5, Informative)
Or lets do it this way. Hoover Dam at peak output produces 2 Gigawatts of power per hour. 11 million servers consume 61 billion KW hours annually. It takes Hoover Dam 30,000 hours (about 3.5 years) to produce that much power. So you need four Hoover Dams just to power all the data centers in the US.
Parent
Re: (Score:3, Informative)
Re: (Score:2)
Units -arghhhh! (Score:3, Informative)
Re: (Score:2)
Re: (Score:3, Funny)
cogeneration (Score:3, Funny)
Or better yet! DatacenterBurgerKing with CPU-broiled whoppers.
Re: (Score:2)
Re:cogeneration (Score:4, Interesting)
Another question, why do we vent the exhaust from our refrigerators into the house during the summer? Just seems like there's a lot you could do to save energy just by moving what would outerwise be waste heat to places where it can either be used or at least not cause a larger cooling problem.
Parent
Guessing (Score:4, Interesting)
Climate controlled. There's this element among building planners that think any outside air is bad(TM). That's why, even in small buildings where you don't have to worry about pressure differentials blowing windows out like you do in skyscrapers, you can't open a frick'n window in the Fall or Spring when the air smells wonderful and there's this perfect chill in the air the just stimulates the brain.
I'm drenched in sweat here in Hotlanta (it's 82F and 66% humidity and climbing to 94) and I really miss New England's Spring and Fall.
Parent
Um.. this is happening (Score:2)
Re: (Score:3, Insightful)
Pollution is easily taken care of with a filter. Controlling physical access is trivial. Humidity may be a bit more involved, but then again, you're heating the incoming air which reduces its relative humidity. Condensation isn't likely. If it does turn out to be a problem, use a heat exchanger and preheat the incoming air using the exhaust air.
But it is not terribly practicle to plug the plenum passages once a year.
So? Inst
Data Center Jacuzzis (Score:2)
great news for Sun (Score:3, Informative)
Disclaimer: I own a tiny bit of Sun stock. (But I bought it because I believe in them, not vice versa!)
Simple Solution (Score:5, Insightful)
Further, the cost to handle each extra watt is multiplied thanks to cooling, power back-up, wiring, etc., while increasing the physical size of the building, constructing more datacenters, etc. is just a flat (linear) cost, and mostly just a one-time expenditure at that.
This strange arrangement is what has led us here. It's not the natural evolution of technology to cram as much power consumption into as tiny a box as possible. It's an artificial need, created by the idiotic distribution of fees common to datacenters.
If a few large datacenters declared their fees as a small $$$ value for each unit of space, and additionally a few dollars, per watt of power consumption, you'd see the problem naturally fix itself, through normal economic forces. As soon as watts are the defining factor, companies won't pay more for a cramped 1U server rather than an (inexpensive) 2U or 3U server. You will also see companies happy to pay more for lower-powered server hardware, as having them directly bear the energy cost will make buying efficient servers a significant savings to them.
Re: (Score:2)
Combine these all into a neat little sum on someone's bill.
Try to think of data center 'floor space' as the main stage. Everything is built around maintaining and supply
Re: (Score:3, Interesting)
Of course the (average) price of electricity is figured into it. That is the PROBLEM.
It is a (self-perpetuating) prisoner's dilemma. The more power consumption you can squeeze into the smallest space, the better of a deal you get. Since it's all averaged out, those using more power than average are getting subsidized by those w
Re: (Score:2)
Like shipping packages where the fees are combination of volume, weight, distance, etc. Data center pricing is typically priced based on a combination of space, power, power density, circuits, contract length, continguous space, etc.
Re:Simple Solution (Score:4, Informative)
Parent
Re: (Score:2, Insightful)
Re:Simple Solution (Score:4, Insightful)
Because when you run a multi-million dollar data center, you clearly can't afford install a few-hundred dollar device in each customer's rack especially if it's a major part of how you bill your customer.
Look, the power companies do exactly what the parent poster suggests. Imagine if power companies charged a flat rate each month based on the square footage of your house. There would no incentive (unless your a save-the-planet hippie type which isn't a bad thing) to turn up the setting on the air conditioner (or turn it off all together), keep incandescent lights running 24/7 along with the giant plasma TV. This is essentially how data centers operate today. There is no motivation to have energy efficient servers unless you're the one that owns the data center and pays the power bill. Today the best a data center owner can do is invest in more efficient cooling systems and that's about it.
Parent
Congress will act (Score:5, Funny)
Virtualization? (Score:3, Insightful)
To me, this seems like one of the more important aspects of power efficiency. Individual server efficiency is important, but the gains from higher utilization could be even more significant. Adding another core to a hypervisor will always be more efficient than adding a new system (CPU, Power Supply, disks, video, etc..). The energy efficient hardware can also be applied to the hypervisor hosts. Build efficient servers, and use as few of them as practical.
Many data centers are already greatly decreasing their server count using virtualization. This should be part of any data center energy efficiency discussion.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:3, Informative)
I'm not sure EPA is the right party to be advocating virtualization. The EnergyStar ratings and utility-level programs
location (Score:2)
Get rid of the AC DC power supplys and replace.... (Score:2)
Re: (Score:3, Informative)
Server "sleep" mode? (Score:2)
1) With multiple front-end servers behind NLB, make the NLB smart enough to put some servers to sleep when their processing isn't needed and wake-on-lan those servers (or the equivalent) when they are needed again?
2) Do servers do "speedstep" like desktops/notebooks where the processors and other components go to lower power level modes when they are not being fully utilized? If not, they should enable that.
Re: (Score:2)
Re: (Score:2)
Federal Guidelines for Clock Speed Limits (Score:4, Funny)
Higher voltage (Score:3, Informative)
I have found that stock switching power supplies as found in common computers are slightly more efficient when powered with 240 volts rather than 120 volts. Some more so and some less so. And virtually all of them can be changed over to 240 volts (having the correct 2-pole switching).
And by using 240 volts instead of 120 volts, you can run twice as many computers on the same power loss in the building wiring (same current, same size wire, same power loss due to heat, serving twice the load).
Direct DC fed power systems may or may not provide realistic savings. DC introduces new electrical safety challenges and costs (electrical arcs inside switches, circuit breakers, and fuses, cannot be cut off by AC's zero voltage crossing that DC does not have). This requires lower voltages for equivalent interruption safety. But if power supplies end up losing less power than the building wiring at the higher current, then DC may be the better choice.
We will need more in-depth study to determine if DC will save power or not at a given installation (it may at some and not at others). But for most installations, going from 120 volts up to 208 or 240 volts (depending in which is available) is as simple as rewiring the system (using 2-pole breakers ... requiring double size power panels) and verifying the computer power supplies are ready for the higher voltage.
208 volts is the likely line-to-line voltage in data centers powered by 3-phase (208Y/120) power in North America. Future data centers could be designed for a 416Y/240 volt power system which can also be used to power fluorescent lighting.