How Internet Data Centers Waste Power 170
Rick Zeman writes "The New York Times has extensively surveyed and analyzed data center power usage and patterns. At their behest, the consulting firm McKinsey & Company analyzed energy use by data centers and found that, on average they were using only 6 percent to 12 percent of the electricity powering their servers to perform computations. The rest was essentially used to keep servers idling and ready in case of a surge in activity that could slow or crash their operations. 'Worldwide, the digital warehouses use about 30 billion watts of electricity, roughly equivalent to the output of 30 nuclear power plants.' In other words, 'A single data center can take more power than a medium-size town.' This is the price being paid to ensure everyone has instant access to every email they've ever received, or for their instant Facebook status update. Data Center providers are finding that they can't rack servers fast enough to provide for users' needs: A few companies say they are using extensively re-engineered software and cooling systems to decrease wasted power. Among them are Facebook and Google, which also have redesigned their hardware. Still, according to recent disclosures, Google's data centers consume nearly 300 million watts and Facebook's about 60 million watts. Many of these solutions are readily available, but in a risk-averse industry, most companies have been reluctant to make wholesale change, according to industry experts."
A sales pitch and a loaded gun (Score:3, Interesting)
"Buy our product or we'll agitate for standards that make them mandatory." It's shit like this that annoys me mightily about the NYT.
On a small-scale, virtualize (Score:2, Interesting)
Using VMWare or other similar technologies, you can dramatically cut the amount of the energy you need to power your servers. You can even take advantage of on-demand servers, so that if you do suddenly become busy, it'll power up more hardware to handle the load. Great for optimizing around a 9-5 workday.
Automatic provisioning? (Score:4, Interesting)
I wonder if the excess servers could be left off, and during rush periods, they could be turned on via IPMI [wikipedia.org]?
Re:So? (Score:5, Interesting)
Worst case, if we just include first world people, it's only about a 100W per person
Rough engineering estimate, a watt continuously is a buck per year.
For commercial I'm completely unimpressed. That's like the depreciation on my desk and chair, or the dept "free" coffee budget for a month. A tiny fraction of the overhead lighting power, which is a tiny fraction of the HVAC power, which is a tiny fraction of my salary. In terms of environmental degradation, the gasoline I burn to commute is worse than my share of the corporate data center (based on a KWh being about a pound of coal, so 16 pounds of coal per week, and commute four times per week is about 4 gallons or about 24 pounds of gasoline)
For residential I'm amazed. They need to make $100/yr off my mom who doesn't even have internet access just to pay the electrical bill. I donno if they can make $100 of me per year and I'm always on the net doing "stuff". One interesting comparison WRT advertising is "one million page views per year = one thousand dollars per month or about a penny per pageview". Donno how true that is anymore. But it would imply that just to pay the electric bill the average person would have to visit 27 web pages per day, every day, which seems pretty high across an entire nation.
Sad sad article... (Score:2, Interesting)
There are so many aspects left unexplored. Part of the problem is that power is also wasted on inefficient code. Bad abstractions and poor data structures. Reasons schedule pressure and untrained monkeys doing coding in PHP. It is too much focus on the ones running the data centers, part of the problem is who they are buying their software of.
Re:Automatic provisioning? (Score:5, Interesting)
That's already how many datacenters do it. Still, that capacity takes about 2-10 minutes to come up to speed so you still need somewhat of a buffer. What they need is instant-on servers which the big guys are experimenting with but the problem is not Google or Netflix or Facebook, they run a pretty efficient operation, it's the rest of the business world who'd rather buy an IBM or HP honking piece of metal that converts 20% of it's power to heat before anything remotely useful has been done than experimenting with what they need and could do to improve on such designs.
Re:not bad (Score:4, Interesting)
that's how they're designed (from someone who's designed and executed datacentre solutions). I got out of the game not long before the AMD Opteron 4100 series came out (mid 2010), but at 5.83W per core they're a pretty damn smart solution even by current standards. You're talking about server power consumption of WAY LESS than .001W per request. Probably 5,000 requests are processed before the thing drinks a Watt. If my Atom-powered netbook could handle that kind of workload I would be well happy.
There's nothing wrong with datacentres sitting idle, the "wrong" comes into it when people burn 500W on a PC with 19" monitor just to scroll down Facebook.
Re:A sales pitch and a loaded gun (Score:5, Interesting)
I don't really understand this hostility. I read the New York Times online, everyday. I don't get a paper delivered to my door. Those few, those happy few who actually read this new york times article, read it online.
The circulation is a million pulp, half a million online.
Virtualization (Score:5, Interesting)
This is one major reason that companies (even very large companies with "money to spare") are moving towards virtualization with incredible speed.
I'm not going to go digging for numbers right now, but the statistics show that something like 100 percent of Fortune 100 companies use virtualization, and perhaps 85-90% of Fortune 500 companies.
The larger virtualization solutions will actually take the servers that are idle, migrate them to another host machine, and power down/suspend the "extra" machine(s) that was/were being used during their core business hours.
Virtualization also allows for spikes in cpu/network, and then can take that power back when everyone goes home (a print server, an intranet web server, a domain controller, etc). So, physical machines actually DO get turned off when they aren't being taxed, and with more and more "software defined networking" the interconnects between systems can be scaled and moved also.
Now, I don't know how the big players are using this (e.g. Amazon, VMware, Rackspace, Google). I can't see inside their datacenters, but one would think that something like AWS would have a huge stake in saving power by turning off idle instances and moving VMs. Not only for the power savings from the server directly, but for the (approx) 30-40 percent more energy that it takes to cool the physical machines.
It's also worth noting that larger companies are putting their datacenters in areas with plentiful (cheap) power. Places like Washington state, with hydroelectric power and a cooler average ambient temperature, allow for a huge savings on power right off the bat. Add things like dynamic scaling of server and network hardware, lights-out datacenters, and better designed cooling systems (look at Microsoft's ideas), and there is a huge power savings across the board.
How much energy does the NYT use to print paper copies of the newspaper, distribute and deliver them, harvest the trees and process the paper? Now compare that with the energy that the online NYT uses. Which allows for more people to view the publication for less energy? I'm positive that it is the electronic version.
Re:So? (Score:4, Interesting)
The Earth receives 170PW of energy from the sun. The sun's total output is 380YW (trillion trillion Watts). How much of that we can capture and use is limited mainly by how much money we spend. So I would say that measuring energy with money makes perfect sense.
Re:A sales pitch and a loaded gun (Score:4, Interesting)
Yeah. This article struck me as particularly whiny. 30 Nuclear Power Plants! The horror.
It's almost like they want you to read a paper newspaper or something.
I question virtually ALL the claims in the story.
Its nonsense of the highest order, with no research to back it up. Do you see Google or Amazon publishing utilization rates of server farms?
Do you see Amazon or Google or any cloud provider having problems paying the power bill?
Did they not say that "Data Center providers are finding that they can't rack servers fast enough to provide for users' needs"?
If the power bill is paid, what is the problem?
Why isn't the harm done to the world's resources (and society in general) by publishing the New York Time evaluated?
Nancy Nielsen, a spokeswoman for The New York Times Company, said [nytimes.com] only the limited supply of recycled paper constrained the company from using more of it. She said 6.5 percent of the newsprint used by the company contained recycled fibers.
...
''The inventory of waste newspaper is at an all-time record high,'' said J. Rodney Edwards, a spokesman for the American Paper Institute, a trade organization. ''Mills and paper dealers have in their warehouses over one million tons of newspapers, which represents a third of a year's production. There comes a point when the warehouse space will be completely filled.''