How Google Cools Its 1 Million Servers 87
1sockchuck writes "As Google showed the world its data centers this week, it disclosed one of its best-kept secrets: how it cools its custom servers in high-density racks. All the magic happens in enclosed hot aisles, including supercomputer-style steel tubing that transports water — sometimes within inches of the servers. How many of those servers are there? Google has deployed at least 1 million servers, according to Wired, which got a look inside the company's North Carolina data center. The disclosures accompany a gallery of striking photos by architecture photographer Connie Zhou, who discusses the experience and her approach to the unique assignment."
Re: (Score:2)
The same way as everybody else. (Score:2, Funny)
Take the heat you produce, and dump it somewhere else.
Be nice if it could be my house! I want to avoid turning the heat on till Thanksgiving if possible, but it's getting a bit tough.
Re: (Score:3, Insightful)
Take the heat you produce, and dump it somewhere else.
Sure but there are different ways of doing it.
Google says they have the cold air come up from their raised floor.
Facebook does it differently- the cold air drops down:
http://opencompute.org/2012/08/09/water-efficiency-at-facebooks-prineville-data-center/ [opencompute.org]
I'm no data center engineer but the Facebook way makes more sense to me.
Re: (Score:3)
I'm no thermodynamic engineer, but it seems to me facebook's way would lose out on the fact that the hot air would mingle with the cold air on the way down and warm up a little
Re: (Score:2, Funny)
Re: (Score:1)
Hot Aisle Containment. Without it, yes, the cold air would be mixing with rising hot air.
Re: (Score:2)
For office or building heating, Facebook's method is completely conventional - A/C comes from the roof tiles and dribbles down to the floor which mixes with the existing warmer office air (which is why A/C is on - in a lot of places, you use A/C year round), creating a somewhat even temperature vertically (the ceiling is not appreciab
Re: (Score:2, Insightful)
Not particularly, either to the different ways, or the Facebook way.
Heat naturally flows up, cool air dropping down would fight that ventilation effect.
Re: (Score:2)
indeed and the heated air raising up and being ventilated out will cause lower pressure -> ie. draw the cold in as well :)
Re: (Score:1)
Google says they have the cold air come up from their raised floor.
Google uses a different approach.
Google realized that the so-called cold aisle in front of the machines could be kept at a relatively balmy 80 degrees or so—workers could wear shorts and T-shirts instead of the standard sweaters. And the “hot aisle,” a tightly enclosed space where the heat pours from the rear of the servers, could be allowed to hit around 120 degrees. That heat could be absorbed by coils filled with water, which would then be pumped out of the building and cooled before being circulated back inside. Add that to the long list of Google’s accomplishments: The company broke its CRAC habit.
They also might not have a million servers,
a tiny embossed plaque that reads july 9, 2008. google’s millionth server. But executives explain that this is a cumulative number, not necessarily an indication that Google has a million servers in operation at once
Re: (Score:1)
Google realized that the so-called cold aisle in front of the machines could be kept at a relatively balmy 80 degrees or so—workers could wear shorts and T-shirts instead of the standard sweaters. And the “hot aisle,” a tightly enclosed space where the heat pours from the rear of the servers, could be allowed to hit around 120 degrees.
Our Los Angeles DataCenter does similarly; I haven't had to wear a sweater in a datacenter in at least twelve years. We use air, though, rather than water since we do colocation and a water-cooled system isn't as friendly to standard racks and servers which clients often want to install.
We benefit from the Los Angeles outside air temperature; we don't need to run chillers unless the outside temperature is over 72 degrees, and while it's often over that during summer days it's almost always below that at nig
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
Figure out a way to efficiently transport low-grade heat long distances (or even extremely efficiently transport it short distances), and you might not get a Nobel prize, but you could get quite wealthy, and even feel good about all the energy the world saves.
Until it's revealed that the materials used cause cancer in cute puppies, but, eh, maybe they won't find out until you're d
Re:They should use the waste heat (Score:4, Informative)
I always thought (Score:5, Funny)
they use the chilling effect from all those DMCA notices they receive.
Google faked some of the pictures (Score:4, Interesting)
Google faked at least one picture. Take a look at this picture. [google.com]
The left-hand side is exact copy of the right-hand side. Take a look at the details: The halos from the lights and the texts in the white labels.
Re: (Score:3)
The grill near the first triangular shadow at the bottom, is far more lit on the right. The triangular shadow is also different. However, those labels do seem far too similar...
Re:Nah (Score:5, Interesting)
I’m obsessed with everything being symmetrical for all my work, so I cloned over the left servers to the right side. It just bothered me that there would be a hole when usually servers would be there. I wanted it to look beautiful, and symmetry is beautiful to me.
Re: (Score:1)
No, they have an OCD photographer:
I’m obsessed with everything being symmetrical for all my work, so I cloned over the left servers to the right side. It just bothered me that there would be a hole when usually servers would be there. I wanted it to look beautiful, and symmetry is beautiful to me.
oh, so its not fake because .. the photographer faked it?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
He's not OCD.
I think it depends on perspective.
or do what I did, and open two copies of the jpg, flip one horizontally, and flip back and forth between the images.)
Err nevermind I understand now.
Re: (Score:2)
Re: (Score:1)
Not only are the similar, they're mirror images.
Why put [label] [barcode] (left) on half your servers, then [barcode] [label] (right) on the other half?
Also, the UPSs (or whatver they are, grey box on top) are mirrors (do they make left-handed and right-handed ones?)
Perhaps more damning, the bus-bar power take-offs, along the top (middle). Again, left handed and right-handed ones (power cutoff switch).
Re: (Score:1)
Re: (Score:2)
I agree it's a been flipped (the labels make no sense otherwise), but that grill is different (or my eyes are going in my old age of course).
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Because as the wiggle-gram somebody posted shows, one side has been set slightly lower than the other when copied.
Re:Google faked some of the pictures (Score:5, Informative)
Google faked at least one picture. Take a look at this picture. [google.com]
The left-hand side is exact copy of the right-hand side. Take a look at the details: The halos from the lights and the texts in the white labels.
If you read the link with the interview with the photographer you'll find that she's into heavy post-production editing. Arguably, *all* of the images are "faked" to some extent. She takes many shots of each scene and layers them together selectively to get the effect she wants. She clones out stuff she doesn't want (e.g. she mentions removing an exit sign) and clones in stuff she feels is needed to make the image symmetric, and therefore more beautiful. She doesn't worry about barrel, pincushion and perspective distortion in the original shots and does heavy correction of the final images to straighten the lines and make the angles pleasing to the eye. She shot almost all of the images with long exposures in a darkened room, which makes the relatively small LEDs appear to glow intensely and makes their cast light powerful enough to be very visible when in reality it's not very visible at all.
In short, she's interested in beauty more than in fidelity, and does whatever it takes to achieve it. Personally, I think her results are fantastic.
Oblig: Series of tubes (Score:5, Funny)
Re: (Score:2)
With... (Score:1)
With 1 million interns from India that wave their arms really fast!
Immersion Would Be Better For the Environment (Score:4, Interesting)
Comment removed (Score:5, Informative)
Re: (Score:2)
Re: (Score:2)
As for the slippery floor hazard, I've recently been in a vegetable oil recycling plant and they use a floor coating that resembles sand paper so people don't slip on greasy areas.
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Why not build specific cases that fit perfectly, in a single mold? Then you could make a group of 4, and submerse that (you can even leave the top open and connect a tube in the bottom to create a current inside and turn it into a huge heatsink too). It won't be perfect obviously, but would work better than tubes I reckon, and since you could have the submerse them compeletely, but put them in containers that are open at the top serviceability would be easier, at least.
Re: (Score:3)
Re: (Score:3)
Re: (Score:2)
I don't see how this would be any dif
Re: (Score:2)
If Cinderella's coach was going to turn into a pumpkin, it would surely have already happened before the clock struck twelve.
You're suggesting that at the exact moment when it becomes effective to do so, all the Google data centers turn into immersive pumpkins, so the non-existence of such is a perfect correlate with its viability, since any friction around the decision is inconceivable.
What did you
Re: (Score:2)
Can't read the article ... (Score:1)
... and risk damaging my belief they use ice-pops.
-Oliver
Hot aisle containment (Score:1)
Re: (Score:3)
Re:Hot aisle containment (Score:4, Informative)
Re: (Score:2)
Their cold air is essentially room temperature, as they're using 80 degrees (presumably F) for that side.
They are Google. I think it is 80K!
Re: (Score:1)
That's a lot of it... it's easier to contain the heat. In the end, you'll get the same result: a closed loop system. On one side, there's the HVAC unit taking in hot air and pushing out cold air. On the other is the servers pulling in the cold air and venting hot air. The ideal setup ducts *all* of it. But that's ugly, expensive, and tedious to setup and maintain. A hot aisle system is the best compromise as one's cold air will naturally hug the floor and hot air the ceiling. All you need to do is keep
Re: (Score:1)
A) There are very few people actually within the data center on a daily basis. And when they are, 99% of the time, they're in the cold aisle. (Did you look at their hot aisle? You couldn't walk down it not matter what temp it is.)
B) s/HVAC/CRAC/ then... there's no "H" (heating) involved. And little to no "V", either. And for the record, MANY HVAC systems use chilled water / evap. (hint: the one in this office is an evap unit.) [and many data centers use gycol as it caries more heat than pure water.]
Re: (Score:2)
"...many data centers use gycol as it caries more heat than pure water."
No it doesn't. The heat capacity of ethylene glycol is only half that of water. But mixed with water it acts as anti-freeze, which is convenient since the evaporator that cools the water may reach freezing temperatures occasionally.
Re: (Score:2)
The pictures are very nice (Score:1)
I wish there was a good way to hear the tremendous noise all those fans make in the hot aisle.
Not actually 1M servers according to wired (Score:1)
The was a placard for the millionith server board. Google stated clearly that it was # 1,000,000 and that it did not mean there were 1M in service. That was really clear in the article so the summary messed it up or just missed it.
Zhou manipulated her pics (Score:3)
A Dutch newspaper, which this week published several of Zhous photos, found out - after a thread on Reddit began mentioning possible photoshopping - that, indeed, Zhou manipulated her pictures: http://www.nrc.nl/nieuws/2012/10/20/google-publiceert-prachtige-fotos-van-datacentra-maar-zijn-ze-echt-nee/ [www.nrc.nl] is the link ( story, of course, in Dutch )
When you look at the pictures with a critical eye, you see it quite quickly: on half of the servers, the LEDs are on the wrong side, they are simply mirrored. Zhou declared she is "crazy about symmetry". As one commentor on Reddit put it: "I knew it! For a long time, Google has been trying to make us believe that they have a lot of servers. Well, this proves that they only have very many servers" Google quite quickly admitted to the news, but did not see a reason to take the Zhou series of pictures offline.
where there's movement (Score:1)