Startup's Submerged Servers Could Cut Cooling Costs 147
1sockchuck writes "Are data center operators ready to abandon hot and cold aisles and submerge their servers? An Austin startup says its liquid cooling enclosure can cool high-density server installations for a fraction of the cost of air cooling in traditional data centers. Submersion cooling using mineral oil isn't new, dating back to the use of Fluorinert in the Cray 2. The new startup, Green Revolution Cooling, says its first installation will be at the Texas Advanced Computing Center (also home to the Ranger supercomputer). The company launched at SC09 along with a competing liquid cooling play, the Iceotope cooling bags."
As someone who HAS built & run oil immersed.. (Score:5, Informative)
..computers, allow me to label this a "fad"
The idea is funky, but to get good cooling you want convection (every joule of pump energy from a circulating pump gets transferred into the oil at yet more heat) which means deep tanks which means, to the server environment, goodbye high density.
The ONLY thing that has changed since I was doing this is the affordability of SSDs, which mean that now it is practical to immerse the whole computer, and the mass storage too, which makes things a lot simpler and cheaper, and means you really can be JUST oil cooled, not oil cooled mainly, except for air cooled HDs etc.
TOP TIP from an old hand.
If you are going to oil cool by immersion, buy the latest top quality hardware, because once immersed it stays there, you'll only pull it once to see why it sucks.
BIGGEST mistake experimenters make is using old hardware, cos you always end up playing with it, making mess, ahh fsckit..
Nota Bene if you are building one of these in anger, make allowances for the significant increase in the weight that the oil makes.
HTH etc
Re:Submerged hard disks? (Score:3, Informative)
In the embedded video, they indicate that hard disks need to be wrapped in some material the vendor apparently provides, presumably for just this reason. Not sure how well the wrapping transfers heat.
Re:Forgot to say why I oil cooled. (Score:4, Informative)
Educate thyself: http://en.wikipedia.org/wiki/Mineral_oil#Mechanical.2C_electrical_and_industrial
Just because something CAN burn doesn't make it dangerous to have around potential sources of electrical arcing. Hydrocarbon petroleum products present no real fire/explosion danger unless the substance is warmer than its flash point, which is the temperature above which the liquid substance can evaporate into the air. Below the flash point temperature, oil is only as flammable as plastic. The evaporated fumes mixed into the air are the ignition danger, not the liquid itself.
This is because ongoing hydrocarbon combustion requires steady supplies of freely-mixing HC and oxygen. Sustaining the reaction requires the input of a tremendous volume of oxygen (compared the the liquid fuel volume, anyway), and the oxygen has to get rapidly mixed with the HC. That mixing can't happen quickly enough to the liquid HC. That's why the flash point is such an important consideration--the gaseous HC fumes mix quite well and quickly with atmospheric oxygen, creating nice conditions for a sustained combustion (a fire).
This is even true of gasoline (flash point = -40F). If you pour gasoline into a pail in the middle of a bad Antarctic winter, and you throw a match into the pail, the gasoline will just extinguish the match like a bucket of water.
Of course, if you mix liquid HC with liquid oxygen, or any other eager oxidizers, all bets are off. That shit will explode at cryogenic temperatures if you just look at it funny. (That's how rocket engines work.)
Re:Mainframe (Score:1, Informative)
Ah yes, the good old days...
As I remember it, there were a couple of levels of coolant that were used to cool off a mainframe - some mystery liquid was pumped around through tubes that would flow by the chips needing cooling- it had all the necessary qualities, including being non-conductive in case of a leak. Then that liquid was pumped through a heat exchanger where the heat would get transferred to distilled water which was then pumped to some cooling unit (up on on the roof in our case).
I still remember when the level of distilled water got low - you'd open up the mainframe enclosure and take your gallon of distilled water from the grocery store and pour it in. Very, very weird.
The idea is interesting, but data centers are also about density. These horizontal enclosures seem to waste a lot of room above the rack. And one rack must easily weigh a couple of tons. Imagine having to drain one of those things... I really don't see this taking off... C
Re:Mainframe (Score:1, Informative)
The old water cooled mainframes (IBM, Amdahl, and their ilk) weren't immersion cooled. Some of them ran the water into heat sinks that covered the chips, some used, for lack of a better term, 'water columns' - the side panels of the chassis were hollow and had (cold) water flowing thru them. They used conduction from the board to the side panels - the style varied depending upo the vendor. Remember, these systems predate the single chip CPUs; it took multiple boards to make a single processor so there was quite a bit of real estate that had to be cooled.
The Cray-1 and X-MP systems did this using freon instead of water. The customer provided chilled water that went to a heat exchanger; the other side was freon. The columns in a Cray were hollow aluminum, and modules were a sandwich - a copper plate between two boards. the plate slid between the columns and was mechanically clamped; heat conducted from the boards to the plate to the side columns, into the freon, and out to the heat exchanger. The Y-MP, C90, and T3-D/E systems used the same hollow columns, but the module sandwich used a hollow aluminum boards instead of a solid one. Hoses (that had to be disconnected when you pulled a module) connected the module sandwich to the side columns, freon flowed thru the entire assembly and carried the heat out.
Cray did come out with a line of smaller air-cooled systems (EL, J-90, et. al.), by then they had the processor down to a single chip.
The Cray-2 and T90 systems were immersion cooled. The same method was used for the Cray-3 (not a Cray Research design; Seymour left to form Cray Computers to do this system). Immersion cooling meant that you had a tank big enough to hold all of the liquid in the system (the waterfall, for those that remember the pictures). For maintenance, you pumped the florinert completely out of the system into the tank, popped the panels, and did your module swaps. When florinert overheats it decomposes to cyanide gas so all of these systems had ventilation systems that vented to a safe location; if you suspected a module failure might have caused an overheat you used a cyanide detection kit and ran the ventilator...