Intel Embraces Oil Immersion Cooling For Servers 230
1sockchuck writes "Intel has just concluded a year-long test in which it immersed servers in an oil bath, and has affirmed that the technology is highly efficient and safe for servers. The chipmaker is now working on reference designs, heat sinks and boards that are optimized for immersion cooling. 'We're evaluating how (immersion cooling) can change the way data centers are designed and operated,' said Mike Patterson, senior power and thermal architect at Intel. 'I think it will catch on. It's going to be a slow progression, but it will start in high-performance computing.' Intel's test used technology from Green Revolution Cooling, which says its design eliminates the need for raised flooring, CRAC units or chillers. Other players in immersion cooling include Iceotope and Hardcore (now LiquiCool)."
Wait, isn't oil flammable? (Score:3, Funny)
Is it really a good idea to put computers and hydrocarbons that closely together?
What if there's a fire?
Re:Wait, isn't oil flammable? (Score:5, Funny)
Is it really a good idea to put computers and hydrocarbons that closely together?
What if there's a fire?
Most people would put it out. What, exactly, were you thinking?
Re: (Score:3)
Re:Wait, isn't oil flammable? (Score:4, Insightful)
Yes. I remember on of the main threats to submarines being fires....
Re:Wait, isn't oil flammable? (Score:5, Informative)
Yes. I remember on of the main threats to submarines being fires....
Wait, were you being sarcastic? The number one main most dangerous thing about being on a submarine is a fire breaking out [wikipedia.org].
Re:Wait, isn't oil flammable? (Score:5, Insightful)
Plenty of motorcycle engines are also air/oil cooled. The concorde even used JET FUEL for cooling.
Without exposure to air (to enable combustion to happen), and a temp approaching the flashpoint of the fuel, there's no fire risk.
Re:Wait, isn't oil flammable? (Score:5, Interesting)
Re: (Score:3)
Not really surprised. I worked in heavy industry back in the early 2000's, and we'd immerse our relays and some of our transformer blocks in oil to keep them cool, especially if they were going to very hot parts of the world. In some cases we'd fully skip the electronics, and go right to plain relays for the setups if they were going anywhere where: Power/voltage issues were going to be a problem. Or where brownouts/spikes were going to be a problem. Or where contamination would be a serious issue.
The m
Re: (Score:3)
A word of warning: the oil in the tub was almost certainly a full load of pure PCBs [wikipedia.org]. They were ideal for the purpose: highly dielectric, nonflammable, stable. They're great so long as you look past the fact that they're horribly toxic and carcinogenic, so they were very widely used in exactly that kind of application. Be careful if you ever have to crack one open again.
Re: (Score:2)
Most people would put it out. What, exactly, were you thinking?
Maybe tossing a bag of fries into the hot oil?
Not all oils are flammable (Score:5, Informative)
One example of non-flammable oil is Silicone Oil
http://en.wikipedia.org/wiki/Silicone_oil [wikipedia.org]
Re:Not all oils are flammable (Score:5, Informative)
One example of non-flammable oil is Silicone Oil
You don't even have to go non-flammable - large transformers that you might see next to buildings have been using oil as a coolant and insulation for decades.
http://en.wikipedia.org/wiki/Transformer_oil
Re: (Score:2)
One example of non-flammable oil is Silicone Oil
You don't even have to go non-flammable - large transformers that you might see next to buildings have been using oil as a coolant and insulation for decades.
http://en.wikipedia.org/wiki/Transformer_oil [wikipedia.org]
They also explode and catch fire every now and then. Of course, they are also carrying high voltages and statistically it is pretty rare.
Re: (Score:3)
Re:Not all oils are flammable (Score:5, Funny)
And now you have eleven posters of Tesla in your office?
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Diesel is an oil product that isn't flammable... unless you compress it. Even crude oil is quite hard to light and requires addition of quite some energy to get it lighted, gasoline and certain other oils are the very flammable ones but there are quite some oils that would be considered inflammable under the conditions in a computer.
Re:Wait, isn't oil flammable? (Score:5, Informative)
In a Diesel engine, compression heats it, and it ignites. But compression is not the only way to ignite it.
Re: (Score:3)
Re:Wait, isn't oil flammable? (Score:5, Informative)
In general, you need a wick. Diesel won't burn by itself, unless the temperature is very high or the air superoxygenated.
The same is true for gasoline, by the way. I used to fill a bottle cap with gasoline, and stub out my cigarette in it. It never caught on fire. The "no smoking" rule of gas stations is mostly because of people using open flames to light their cigarettes and pipes in a fume filled environment, and not so much a cigarette that isn't hotter than many exhaust pipes.
Re: (Score:2)
Or unless it is atomized, for example in a crash.
Re: (Score:3)
Were you all out of water?
Re:Wait, isn't oil flammable? (Score:5, Informative)
This is a bit confused, I'm afraid. Diesel has a flash point of 100 to 200 Farenheit depending on the type of fuel, etc. If you get it that hot, or hotter, it can accumulate enough flammable vapor to burn.
You need a certain ratio of vapour to air, and you still need something to ignite that vapour mix.
In a Diesel engine, compression heats it, and it ignites. But compression is not the only way to ignite it.
This is misleading at best, reading like you compress the diesel. You don't - liquids don't compress.
You compress air, which heats to a heck of a lot more than 200F (more like 1000F), and also puts more oxygen per volume in the chamber. The high temperature of the air combined with the high O2 level allows the combustion to take place.
Re:Wait, isn't oil flammable? (Score:4, Informative)
Re:Wait, isn't oil flammable? (Score:5, Interesting)
The liquid part isn't what does the burning. It's the vapor.
Re: (Score:2)
Diesel burns quite well. You just need something like a "wick". Even salad-oil burns that way. And once it is heated up a bit, it does not need the help anymore.
Re: (Score:2)
Diesel burns quite well. You just need something like a "wick". Even salad-oil burns that way. And once it is heated up a bit, it does not need the help anymore.
Unless you pump in a lot of air or oxygen, it still needs a wick.
However, soot can act as a wick too (which is why a candle can burn for so long).
Re: (Score:3)
Mineral oil is combustible in the same way that wax, sugar, wood, etc, are. It burns, but it isn't flammable, as the flashpoint is way too high.
You need to hold it at a high temperature to sustain combustion (like a wick, for example).
Re: (Score:2)
Re: (Score:2)
Is it really a good idea to put computers and hydrocarbons that closely together?
What if there's a fire?
It works really well if you keep the oxygen out of the system or if the oil vapor pressure is low enough that it doesn't create a flammable headspace.
Re: (Score:3, Funny)
Re:that;s why I use carbohydrates (Score:5, Funny)
No, he's suggesting beer should be earmarked as 'high performance server coolant', the keg as a 'coolant storage reservoir' and the tap as a 'used coolant bleedoff valve', the latter to be placed in the bofh's office next to the coffee machine.
Re: (Score:2)
Re: (Score:3, Insightful)
Don't worry. It's Coors... Not really suitable for human consumption.
Re:that;s why I use carbohydrates (Score:5, Insightful)
Don't worry. It's Budweiser ... Not really suitable for human consumption.
FTFY
Re: (Score:3)
Don't worry. It's Fosters... Not really suitable for human consumption.
FTFY
AFTFY
Re: (Score:2)
I'd worry more about the tendency of oil to dissolve various substances, and also evaporation.
Re: (Score:2)
Liquid Metal CPU cooler (Score:2)
This brings back good memory for the liquid metal CPU cooler that I used a while back
A review is at http://www.guru3d.com/article/danamics-lmx-superleggera-review/ [guru3d.com]
Unfortunately the vendor already closed its doors, or I would have bought more coolers from them
Re: (Score:2)
If you have to build a custom case because your cooler is a giant turbine, you might as well go with immersion as so many others have in the past. [google.com]
Re:Liquid Metal CPU cooler (Score:5, Informative)
The cooler uses a NaK alloy. Check youtube and you'll see that this reacts violently with water and will ignite in air. The company claims that they wouldn't worry about leaks in the cooler but I wouldn't trust them or want that stuff near my expensive hardware. The craziness of using NaK alloy as a coolant for a computer is probably why the company closed it's doors.
Re: (Score:2)
The cooler uses a NaK alloy. Check youtube and you'll see that this reacts violently with water and will ignite in air. The company claims that they wouldn't worry about leaks in the cooler but I wouldn't trust them or want that stuff near my expensive hardware. The craziness of using NaK alloy as a coolant for a computer is probably why the company closed it's doors.
Not to mention it is twice as much money as the zalman which outperforms this cooler in every way.
Re:Liquid Metal CPU cooler (Score:5, Insightful)
I wouldn't trust them or want that stuff near my expensive hardware.
Wow, you care WAAAAY too much about your hardware.
I wouldn't tust them or want that stuff near ME.
Screw the computer.
Re: (Score:2)
Holy Christ. That thing is pumping sodium-potassium alloy through your computer. This stuff is used for coolant in some nuclear reactors. I had a passing thought some years ago at how humorous it would be if someone was actually crazy enough to use it to cool a desktop computer. It's a good coolant but it is VERY volatile. I don't think I, or my insurance, would be very comfortable having that in my home.
Danamics had this to say in the guru3d article:
Re: (Score:2)
If a technology company cant even spell leak ( leek ) or moisture ( moist ) then I have absolutely no faith in anything that they do.
If they let simple things like that slide, then who knows what else they have missed.
No thanks.
Re: (Score:2)
It is a Danish company, I would cut them some slack.
Re: (Score:2)
On the forgiveness of the language I do agree, on everything else I would need some convincing (and believe me, I would LOVE something as cool as a liquid metal cooler).
Re: (Score:3)
To ensure cetacean transparency? XD
Re: (Score:2)
Oh no, I'm very aware that volatile concerns how willing a substance is to vaporize. You took the #1 definition from our lord and savior, the dictionary (especially the online version). But you forget the secondary definition:
2.
Tending or threatening to break out into open violence; explosive: a volatile political situation.
Aside from the political example it describes NaK perfectly - able of breaking out into open violence.
Economy of Scale (Score:2)
I hope this stuff hits the discount rack soon.
Not only safe... (Score:5, Funny)
Re:Not only safe... (Score:4, Informative)
I think chunks of fries and burgers (and cell phones and other crap that gets dropped into the oil in restaurants) may cause certain issues with the flow of the heat.
Re: (Score:2)
P'Shaw, that's never stopped McDonald's or burger king!
Re: (Score:3)
Mineral oil is generally used as a laxative, so unless they are cooling with vegetable oil, I'd advise against this. :-p
Perfect (Score:5, Funny)
Have you ever worked at a company where middle management could not have used daily bunches of fries with extra laxatives?
What "News"! (Score:2)
What about the weight? (Score:3, Insightful)
They used to say ... (Score:5, Funny)
I welcome our new UK computing overlords.
Re: (Score:2)
Perhaps they can get Lucas involved. Then it can leak oil and release magic smoke at the same time.
I was considering something like this a few years (Score:2)
I was considering something like this a few years ago. But instead I went with conventional air cooling inside an ornately carved wooden case instead.
Note that capillary action inside the cable tends to create oil drips all over the place unless you inject glue/epoxy into all your cables to seal the the tiny gaps between insulators..
http://www.pugetsystems.com/aquarium_computer/V2/module.php
Pff, that's so 2006. (Score:2)
The first place I ran across the concept was Tom's Hardware, and you can still see the original article. "High Performance Computing" says Intel? Pish Tosh. Kids, you really can try this at home... but get a grown-up to assist you!
http://www.tomshardware.com/reviews/strip-fans,1203.html [tomshardware.com]
Let's party like it's 1999 (Score:3)
http://web.archive.org/web/19991006062047/http://www.accsdata.com/drffreeze/TestBox2.htm [archive.org]
Sadly, all the pictures appear to have been lost.
I remember this guy going through and dunking his systems in Mineral oil over a decade ago, back when I was in 11th grade. You know, back with the BP6 was amazing shit and slotkets were an essential overclocker's tool.
Wow your evaluating (Score:2)
something that was proven to be fine in 1985?
sure it wasnt oil, it was an exotic chemical developed by 3M but the point still stands
'We're evaluating how (immersion cooling) can change the way data centers are designed and operated,'
its been proven on machines that produced much more waste heat than today 27 years ago in the cray 2
not to mention countless people using oil to cool their high voltage transformers and overclocekd P4's, but yay, GO Intel, grats on the prior art, obious patent in the nea
Re: (Score:2)
The cray2 didn't use that much more power than todays high density systems, 195kw in about the same floorspace as four racks, which is inline with high density designs of today. I'd imagine that's why we're looking at similar solutions.
Why don't they put datacenters in cold places? (Score:2)
Re: (Score:2)
Re: (Score:2)
Dew point. Colder than that and you have to deal with condensation.
Oil works well, but current solutions work with total immersion and that's a freakin mess. And seeing the heatsink capacity of water is higher to begin with, warm water cooling is a much more practical solution.
Re: (Score:2)
Re: (Score:2)
You must have missed the last six months worth of news stories about Facebook's selection of their new datacenter. Not only is the year-round temp ~60F, but it's right next door to a series of giant hydroelectric dams in one of the wettest regions of North America.
raising the most important question: (Score:2)
Do you want fries with that?
Re: (Score:2)
If you're in Britain you want a computer with efficient chips.
Re: (Score:2)
there's something fishy about your comment
New problem (Score:2)
Floor loading. How much is a rack of oil filled servers going to weigh?
Raised Flooring (Score:3)
I spent most of the 80's working on flight simulators that had rows of cabinets on raised flooring. One sim was supposed to be at 70F and the temp was usually so stable that if it was up more than a few degrees we could tell by feel and smell as soon as we walked in the room.
By shear luck I worked on simulators in Las Vegas, New Mexico and South Korea, all places that in the summer you really wouldn't want to be working outside. The constant temp during working hours was great ( though I think it made me more of a wimp for temp extremes when I went outside ) Thinking about the oil immersion and what I'd guess would be warmer ambient temps in computer rooms is a little sad. It was the extra cool computer rooms that I worked in that added to the appeal of my job back then.
Two birds with one stone (Score:2)
After the oil has been heated up by all those processors, pipe it to nearby fast food restaurants to cook French fries and all those other delicious, fattening foods! Yum!
1998 (Score:4, Interesting)
hard drives? (Score:2)
"hard drives [...] withstood the oil just fine"
I'd like to know if they used off-the-shelf hard drives for this. I find it hard to believe that a hard drive would work in oil. They usually have breathing holes, wouldn't oil get into the drive and interfere with the moving parts?
Re: (Score:2)
Hard drives need to be sealed to be immersed, as mentioned at http://gigaom.com/cloud/intel-immerses-its-servers-in-oil-and-they-like-it/ [gigaom.com] .
So.... (Score:2)
So, still haven't licked that optical chip problem yet?
mental arithmetic (Score:2)
Glancing through the headlines with divided attention, I mentally juxtaposed the headings of two successive stories, yielding "Iran embraces oil immersion for critics".
80s (Score:2)
http://en.wikipedia.org/wiki/Full_immersion_cooling#Liquid_submersion_cooling [wikipedia.org]
Re: (Score:3)
You realize, of course, that datacenters don't "remove" anything smaller than an entire blade (or depending on the scale involved, they pull an entire rack). Then they rotate a spare into place, ship the bad one out the door, and let the vendor screw around with figuring out "why" it failed.
Intel doesn't mean for your average Mom n' Pop running Windows SBS in a half-rack mounted PowerEdge to use immersion cooling.
Re:It's great until... (Score:5, Insightful)
Oil works great until you have to remove something... You realize, of course, that datacenters don't "remove" anything smaller than an entire blade (or depending on the scale involved, they pull an entire rack). Then they rotate a spare into place, ship the bad one out the door, and let the vendor screw around with figuring out "why" it failed.
I doubt most datacenters swap out racks. Unless they've built a crane into the datacenter, you'd need to get a forklift to move the entire rack and there isn't clearance for that. Swapping blades is entirely reasonable, swapping 1U servers less so unless you have some really smart automation and failover to reimage the server and get it back to the previous state.
Re: (Score:2)
Re: (Score:2)
If you're running a datacenter of any reasonable size, there's lots of common automation that makes pulling a 1u server convenient and easy. At the most basic level, kickstart + puppet + a load balancer makes the process pretty seamless - shut the machine down, and it's automatically removed from the pool. Pulling the box, and re-installing require far less human time than troubleshooting hardware problems.
If you use OSS, that approach is actually simple and free. Cobbler, puppet, and haproxy make this kind
Re: (Score:2)
But why do I keep having to remove line cards from an MLX-16 or PCIe cards...
Re: (Score:2)
Take it out with some gloves, do the repair, put it back. If done properly you would only need to clean up a little bit. You could also simply replace the whole module and ship it back to manufacturer. Even in servers there is little to repair these days, fans, hard drives, anything mechanical is usually the culprit. RAM, SSD, CPU's don't die (after burn-in testing) for decades.
Re: (Score:3)
Ram dies all the time, it's my second highest AFR part after hdd's and ahead of both fans and psu's which are the only other components with a statistically significant failure rate.
2000 do-it-yourself (Score:4, Interesting)
June 30, 2000: Slashdot reports that some overclockers have solved their cooling problem by immersing their motherboard in Fluorinert [slashdot.org]. Crazy kids. Who knew it would eventually catch on?
Re: (Score:2)
Didn't Seymour Cray pioneer the concept?
Re: (Score:2)
Fluorinert is crazy expensive. A liter of the stuff costs around $440~$540
Fluorinert made since with the Cray computers because they were already crazy expensive monsters.
You can get recycled Fluorinert for cheaper, but you're still talking hundreds of dollars for the volume of liquid necessary to submerge a motherboard and provide a useable thermal mass.
Mineral oil is a vastly cheaper alternative.
Now that Intel has done R&D that satisfies their engineers, they can roll this out to the masses at a reaso
Re:Cray did this decades ago (Score:5, Informative)
Not to mention Slashdot's own coverage (possibly incomplete):
2003 [slashdot.org], 2005 [slashdot.org], 2006 [slashdot.org], 2008 [slashdot.org], 2010 [slashdot.org], 2011 [slashdot.org]
Re:Cray did this decades ago (Score:4, Funny)
These low UID users are so old, they've been replaced by Beowulf clusters of nanobots that remember every single Slashdot post ever.
Re: (Score:2)
FTFS
'We're evaluating how (immersion cooling) can change the way data centers are designed and operated,'
immersion cooling is already a proven technology, doesnt matter what the medium is.
Q: What's the best way to get heat out of a CPU? (Score:2)
Re: (Score:2)
Re: (Score:2)
GOD I hate this fucking site
You called?
Oh yes, one would think that most geeks would be quite happy to watch shills fuck, as long as they are pretty enough.
Please close the door after you hand in your geek card.
Re: (Score:2)
Oil immersion works, but why bother? 99.99% of datacenters still rely on moving cool air through hot servers to cool them. in some instances you could make a case for mineral oil bath cooling. if you want to push the envelope of server cooling, try using our Vertically cooled servers. www.cirrascale.com. we believe that hot air naturally wants to rise. we pack 72 18"tall 1U wide servers in one rack, or 96 13" tall servers. 10's of thousands deployed we've been able to cool over 30KW of load in one rack with Air on a non raised floor datacenter. We've been doing it for years, and we don't void warranties to do it. "excuse me, Mr. Dell/HP/IBM service center person... this server isn't working.. can you take a look at it please" "Be happy to... Um uh, why is it in a plastic ziplock bag?" "oh that's to keep the mineral oil from dripping on your service lab floor" RMA declined!, Warranty Null & Void! mark.skinner@cirrascale.com
Sure you can cool that with air but if you're adding an extra 50-60% in energy costs to cool it and using immersion cooling will only add 4% in energy costs, that'll certainly get people's attention. Add in reduced costs for CRACS and other related equipment and immersion cooling looks more viable. Finally, if your OEM provides equipment that is spec for immersion cooling, I'm pretty sure that they'll provide warranty support for it as well.