Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
China Hardware

New Approach To Immersion Cooling Powers HPC In a High Rise 63

Posted by timothy from the smells-like-mineral-oil dept.
miller60 writes "How do you cool a high-density server installation inside a high rise in Hong Kong? You dunk the servers, immersing them in fluid to create an extremely efficient HPC environment in a hot, humid location. Hong Kong's Allied Control developed its immersion cooling solution using a technique called open bath immersion (OBI), which uses 3M's Novec fluid. OBI is an example of passive two-phase cooling, which uses a boiling liquid to remove heat from a surface and then condenses the liquid for reuse, all without a pump. It's a slightly different approach to immersion cooling than the Green Revolution technique being tested by Intel and deployed at scale by energy companies. Other players in immersion cooling include Iceotope and Hardcore (now LiquidCool)."
This discussion has been archived. No new comments can be posted.

New Approach To Immersion Cooling Powers HPC In a High Rise More

New Approach To Immersion Cooling Powers HPC In a High Rise

Comments Filter:

  • Now Hiring: SCUBA certified network admins (Score:2, Funny)

    by Anonymous Coward

    Waiting for a watered-down version for the consumer space.

    • didn't the computers and physchic people in minority report work like this? the future is now! Not to mention individualized advertising in stores, and gestures to control computers. I should rent this movie again... did it have a jetpack scene as well?

      • Re: (Score:1)

        by Anonymous Coward

        Not to mention a mind control device that wrapped around the skull -> Google Glass....

  • Two phase is asking for trouble. (Score:4, Informative)

    by camperdave ( 969942 ) on Thursday November 14, 2013 @12:06PM (#45423700) Journal
    Using a boiling liquid is asking for trouble. True, a phase change like boiling can conduct away a lot of heat, but there are other issues. First of all, vapour does not conduct as much heat as liquid, so there's going to be an insulating layer of vapour over all of the components that need it most. It's called the Leidenfrost Effect [wikipedia.org]. Second, bubbles expanding and collapsing causes an effect called cavitation, which can erode components. It is a constant sonic vibration which can induce metal fatigue in delicate wires (such as the leads inside a chip), and can cause cracks in inflexible materials such as silicon chips and ceramics (capacitors and resistors).

    • Re:Two phase is asking for trouble. (Score:5, Informative)

      by sugarmatic ( 232216 ) on Thursday November 14, 2013 @12:25PM (#45423872)

      Nucleate boiling is what keeps the lights on if you depend on coal or gas for your electricity. It precedes the zone where your Leidenfrost effect is relevant, and actually increases the heat transfer coefficient by factors.

      Tuning a closed system to exploit this is an exercise (fluid chemistry, pressure, temperature), but it is also ubiquitous. As for cavitation, it's a red herring in the nucleate boiling zone- the size of the bubbles is so small, and hence the driving frequency is so high, there is a) less mechanical coupling for the vibration, and b) the energy of cavitation is so low as to not be an issue.

    • I'm not sure that the Leidenfrost effect applies as much when all the components are immersed in liquid. It seems to be more of an issue when a hot surface is suddenly exposed to a small amount liquid. Second the corrosion effects you describe have more of an effect with water and sudden pressure changes. The Novec fluid used doesn't have the same properties as water.

    • Re:Two phase is asking for trouble. (Score:5, Insightful)

      by DerekLyons ( 302214 ) <fairwater.gmail@com> on Thursday November 14, 2013 @12:48PM (#45424072) Homepage

      The Liedenfrost Effect only really causes problems when you have to remove a great deal of heat very rapidly without pumps - I.E. inside a boiler. Inside a PC, it's very unlikely to be a problem. The same goes for cavitation, it's only a problem when there's a large number of bubbles forming and collapsing very rapidly or if there is a high pressure differential. Inside a PC, these conditions are unlikely to obtain and are thus unlikely to be a problem. (Not to mention, current PC's have constant vibration due to fans, hard drives, CD/DVD players, etc... and don't seem to suffer from any problems.)

      • The naive overclock nuts who try liquid nitrogen on either a bare CPU die or a bare heatspreader sometimes discover the hard way that the Liedenfrost Effect is enough to generate a nice cushion of nitrogen between their couple-of-hundred-watts in a square centimeter or two CPU and the bulk of their liquid nitrogen; but that's mostly a surface area problem, which could just be solved with a basic heatsink, were they not trying to cut out every last component in the chain that has a nonzero delta-t.

    • Re: (Score:1)

      by Anonymous Coward

      Which is why heatpipes don't work at all and don't last for decades... oh, wait.

    • How do you explain the major success that is called the heat pipe, then?

      • That effect is called "X, but more slowly." For example: Nuclear explosion, bad. Nuclear explosion, but more slowly, awesome. Rapidly boiling ammonia, bad. Evaporating ammonia with a high partial pressure, good.

      • Easy. The fluid in a heat pipe is not in direct contact with the CPU. The metal of the heat transfer block and pipe is what is conducting the heat away from the CPU. Any Leidenfrost effect within the pipe merely insulates that section of pipe, and the heat simply conducts further down the pipe wall.

    • Re: (Score:3)

      by CODiNE ( 27417 )

      Obligatory link to THE Leidenfrost effect paper.

      EPIC [wiley.com]

      Do Not Try This At Home.

      But if you do...
      Try not to shatter your teeth.

  • Most buildings have a core water chiller. It wouldn't be that hard to have a heat exchanger going to the fluid circulation system, then liquid cool every rack and item inside.

    The big problem is engineering the valves and connectors. It would be nice for a leak or an improper connection to be detected, and a valve shutting off coolant until it is fixed. Having quick-connect connectors which will shut off coolant flow when disconnected is also imperative. The goal would be for an almost immediate disconne

  • Controlling vapor loss? (Score:3)

    by fuzzyfuzzyfungus ( 1223518 ) on Thursday November 14, 2013 @12:15PM (#45423790) Journal
    TFA doesn't say exactly which 'Novec' heat transfer fluid they use; but all of them have enthusiastic vapor pressures even at 25 degrees, and low boiling points. The devices shown in the pictures (unless specially opened just for the glamor shots) also don't look particularly well sealed.

    How do they deal with that? Does the 3M guy follow the milkman every morning, and deliver another couple thousand liters? Is there a chiller/condenser somewhere in the air circulation system that scrubs most or all of the escaping vapor? Are the racks normally sealed tight?

    I, again, couldn't get any solid quotes for medium-large quantities of the heat transfer fluids; but fancy fluoridated-carbon engineered fluids aren't generally cheap enough to just ignore large losses of. Boiling may well be more efficient than pumping as a heat transfer method at the board level; but I'd be amazed if they can get away with running this as anything other than a closed loop, despite the pictures seemingly showing otherwise.

    Anybody know?
    • From the summary

      OBI is an example of passive two-phase cooling, which uses a boiling liquid to remove heat from a surface and then condenses the liquid for reuse, all without a pump

      It's light on details, but indicates that they are somehow reclaiming any boiled off fluid.

    • OBI is an example of passive two-phase cooling, which uses a boiling liquid to remove heat from a surface and then condenses the liquid for reuse, all without a pump.

      At least read the summary of the article.

      • I read that claim in the summary; but was baffled by where and how the condense the liquid for reuse; because the pictures showed no signs of vapor capture or chiller/condenser units(at least nothing that looked similar enough to the ones I've seen for me to recognize them).

        • Re: (Score:1)

          by N2717Y ( 3432178 )
          Are you looking at the same pictures? At least half of the pictures is a condenser.

          http://www.datacenterknowledge.com/wp-content/uploads/2013/11/applied-control-novec.jpg [datacenterknowledge.com]
          • I think those parallel green things are circuit boards boiling the coolant furiously. You can see the twisted wiring bundles running along the bottom edge of the picture, and what look like either interconnect headers or brass standoffs between the boards.

            The copper pipes are obviously involved in circulation in some way; but show absolutely no condensation, nor any fins or other surface-area-increasing features, so I'd be surprised if they are chillers.

            I think that that might be a shot of their FPGA
        • Most of the pictures are of sealed tanks. The picture of fluid boiling is clearly labeled as being an image of what is inside of the sealed tanks. Certainly they have not shown every part of the system, but the photos shown certainly do not contradict the claims of the article.

    • Re: (Score:1)

      by N2717Y ( 3432178 )

      Changing fluid or refilling is not necessary with well designed systems, because they are made to avoid or minimize losses (including vapor) or contamination. We found that spilling liquid while pouring in and out of containers is by far biggest source for losses, and we have integrated ways to avoid spills on initial filling, along with many other technical features that are designed to keep fluid in the tank.

      From Allied Contro's FAQs:
      http://www.allied-control.com/immersion-cooling/frequently-asked-questions-faqs/item/4-do-i-have-to-change-or-refill-the-liquid [allied-control.com]

      The key difference in open bath immersion cooling is to keep tanks under ambient pressure.

    • It looks pretty darn well sealed to me. It's just a thermopile design. Phase change simply means a carefully designed system can get away without pumping a lot of fluid around.

      The stuff is expensive. A gallon or so can set you back several hundred dollars...and not new stuff either....that's a reclaimed cost.

  • Slashdot = fookin' hoors? (Score:1)

    by Anonymous Coward
    So, what, is Slashdot shilling for 3M now or something?
    Liquid cooling of hardware isn't a new idea at all. How is this even a news story?
    Come on /., how much did 3M pay you to whore yourselves out on this one?

  • One thing that was mentioned was removing the air cooling parts from the servers. I'm wondering now if we'll see server vendors putting together configurations that will be specifically designed for immersion technology and if that will improve density / cubic meter in terms of rack space. Wouldn't this lead to higher density configurations, say 2 or 3 times current rack density or more? Also what about disk drives (rotating) can they be immersed?

    • Blade servers are already done in this stripped down format, with the PSU and cooling parts in the chassis. Problem is, they're prohibitively expensive right now. Since these immersed servers wouldn't require cases, I'd imagine they'd cost quite a bit less.
    • Immersed HDDs aren't going to be happy. Barring some (very expensive) edge cases that are carefully sealed with a suitable gas mixture (so that the heads will continue to float above the platters at very high altitudes/low pressures and/or very high pressures won't cause increased drag, a niche that has probably been largely murdered by cheaper SSDs of late), HDDs aren't fully sealed. They have some aggressive dust filters, and the breathing hole isn't large, so infiltration might well be fairly slow; but o

  • Dunk your own computer! (Score:3)

    by bobbied ( 2522392 ) on Thursday November 14, 2013 @12:26PM (#45423884)

    http://www.pugetsystems.com/submerged.php

    I always thought this was a cool idea, albeit pretty messy. But this company will sell you a "case" (a modified fish aquarium) and the components to go into it. They also have some specs on how well this thing cools.

    • Looking at the video the cpu and power supply fans are still attached and spinning. How does this affect the life of the fans as the are pushing around a liquid instead of just air? Do the fans do much as all the components are in the liquid? Does anyone know more about this?
      • The Puget arrangement uses a fluid with a much higher boiling point, so enthalpy of vaporization (and rapid changes in density) aren't carrying heat away as actively as in TFA's boiling coolant arrangement. Given that, I suspect that having fans/propellers pushing hot coolant out of the heatsinks is helpful. Since passive/convection cooling is possible with air, and fluid cooling is likely much more efficient, it'd presumably be possible with fluid as well; but 'possible' and 'easy' are not identical words,

      • Read their information page. The fans in the Oil do turn much slower than in air, but they are really not necessary when the whole thing is submerged in oil. They also have had a system in operation since 2007 with no fan failures, or any other failures. Apparently the oil dissipates any heating effects of turning slow. I wouldn't expect that the fans would ever wear out in this situation. When fans fail, they get noisy and then stop turning. The primary failure mode is the bearings wearing out due to d

  • Coming Soon to a Movie Near You... (Score:3)

    by mythosaz ( 572040 ) on Thursday November 14, 2013 @12:38PM (#45423966)

    ...the bad guy will point his gun at the server room, threatening to shatter the glass case with the server cooling liquid in it, he'll shoot, and use the rushing liquid to block the heroes from stopping his escape.

  • I mean, I imagine solid state storage would be fine, but I can't see a mechanical disk behaving too well in this sort of environment. Unless it's completely watertight, of course. And I don't see disks going away anytime soon.

    I suppose you could have a separate, air-cooled storage bay, but that would introduce new exciting engineering difficulties...


    (To be absolutely fair, I imagine something like this is going to cost a bit, and if you are spending that much already you might as well be springing f
    • Probably cheapest to go with a single, small, SSD onboard, that would then connect to networked storage for actual capacity.

      Going totally diskless, and doing an iSCSI or Fibre Channel HBA, would be more elegant; but you can probably get 128GB of decent quality SSD for less than a bootable HBA of either flavor would cost you, and that's easily enough space to install anything from a VMware hypervisor to a full Windows or Linux server capable of speaking almost any network storage protocol you want through

    • Re: (Score:1)

      by N2717Y ( 3432178 )
      Not only solid state storage, also helium-filled hard drives (such as the WD 6TB Ultrastar He6).
      • Thanks for that reference - I'd never heard of those drives before. Now I've got some learning to do.

  • Other players in immersion cooling include Iceotope and Hardcore (now LiquidCool).

    Who's principle leads are AcidBurn and CrashOverride respectively, no doubt.

  • I wonder how Novec compares cost wise to Fluorinert. We use Fluorinert at work here and I know the larger FC-72 jugs are very costly at around $1500 and the FC-40 jugs are priced as if they were filled with liquid gold, over 3 grand and much smaller than the FC-72 jugs.

Slashdot Top Deals

If you aren't rich you should always look useful. -- Louis-Ferdinand Celine

Close