Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Supercomputing Hardware

Supercomputing Cluster Immersed In Oil Yields Extreme Efficiency 67

Posted by samzenpus from the staying-cool dept.
1sockchuck writes: A new supercomputing cluster immersed in tanks of dielectric fluid has posted extreme efficiency ratings. The Vienna Scientific Cluster 3 combines several efficiency techniques to create a system that is stingy in its use of power, cooling and water. VSC3 recorded a PUE (Power Usage Efficiency) of 1.02, putting it in the realm of data centers run by Google and Facebook. The system avoids the use of chillers and air handlers, and doesn't require any water to cool the fluid in the cooling tanks. Limiting use of water is a growing priority for data center operators, as cooling towers can use large volumes of water resources. The VSC3 system packs 600 teraflops of computing power into 1,000 square feet of floor space.
This discussion has been archived. No new comments can be posted.

Supercomputing Cluster Immersed In Oil Yields Extreme Efficiency More

Supercomputing Cluster Immersed In Oil Yields Extreme Efficiency

Comments Filter:

  • Immersing them in oil is fine... (Score:3)

    by 93 Escort Wagon ( 326346 ) on Monday July 06, 2015 @05:55PM (#50058703)

    ... but rubbing them with cheetah blood makes them even faster.

  • Dedicated, highly trained staff (Score:4, Informative)

    by TWX ( 665546 ) on Monday July 06, 2015 @06:03PM (#50058751)
    This is fine when the application is high-end and the organization can justify employing expensive, highly-trained staff to maintain the equipment, but for the average business, even one with a fairly extensive datacenter, there is no practical way to do this. Too many staff do double-duty (ie, take care of software applications and occasionally do hardware maintenance) or are too reliant on software/systems vendors that are themselves locked in to hardware incompatible with such an environment for this to be practical. It's cool, but it's definitely a niche within a niche.

    • That's right.

      The company you'll see in the 2020 Fortune 250 who deals with all that nastiness is being formed right now because you get the willies.

    • Agreed.

      Still, you have to admit that it brings new meaning to the phrase: "Do you want fries with that?"

    • If quality equipment is purchased and oil cooling is optimal i doubt that internal maintenance will be much of an issue at all. Oil immersed gear if free of dust and runs cool therefore components tend to be very long lasting. No fans or moving parts may be needed at all. I have experience with cooling towers for industrial chillers and the amount of water used can easily exceed the cost of electricity to cool a large building. I am aware of one school that had worn out baffles in an AC cooling

      • Then a hard drive fails and someone has to get very wet and make a big mess to replace it.

      • Re: (Score:2)

        by Rhys ( 96510 )

        Its cute but you're going to get failures. I hope they did some really amazing burn-in tests before immersing. We had 5% defective on arrival ram (ECC issues -- cuts a nodes gflops to 2/3 to 1/2 original) from a certain fruit vendor, using their gear in a supercomputer I ran. Most of the lifetime issues were just drives dieing, but we regularly also lost CPUs, myrinet boards, fan units, and even the occasional motherboard (the NICs tended to die). There was always something new dead every week or two while

  • I use whale oil (Score:1)

    by Anonymous Coward

    I'm all for saving the environment :)

  • Would not oil start dissolving all the parts as it would be harsh on glues and plastics that most components are made up of?

    • Re:Would not the oil start dissolving the parts? (Score:4, Informative)

      by joaommp ( 685612 ) on Monday July 06, 2015 @06:16PM (#50058855) Homepage Journal

      It's an inert type of oil developed by 3M for exactly this purpose.

      • It's an inert type of oil developed by 3M for exactly this purpose.

        In addition to being chemically inert, a good liquid coolant should have other properties:

        1. Electrically insulating (duh).
        2. Low viscosity, so it flows easily.
        3. High thermal conductivity.
        4. High thermal expansion, to increase natural convection.
        5. Low vapor pressure.
        6. Non-flammable.
        7. Non-toxic.
        8. Cheap.
        Liquid fluorocarbons have often been used. But the fluid in this case is not a fluorocarbon. It is specially blended white mineral oil.

        • Re: (Score:2)

          by guruevi ( 827432 )

          Non-flammable, non-toxic and cheap aren't really necessary for these things. Nobody is going to light a flame, drink it and the cost is usually offset by the savings.

          • You want the stuff non-flammable because you'll be sinking electrical equipment in the goo, so sparks need to not make the whole computer room go boom.

            You want the stuff non-toxic because in the event of an inadvertent leak or a disgruntled employee with an axe, you don't create an instant Superfund site. Nor do you want the people who maintain the racks to need to wear hazmat suits.

            • Re: (Score:2)

              by guruevi ( 827432 )

              Sparks in oil? You could soak the thing in diesel fuel or even gasoline, as long as it is fully submerged and contained there is not enough oxygen to ignite it and sparks do not develop in an oil based substance (again, you need oxygen to get a flame).

              I agree it shouldn't be hazardous but "toxic" does not always mean it's a hazard. There are berries that are toxic to humans yet totally fine in nature. Also, there are other containment methods and cleansing procedures similar to how underground gas station t

          • Re: (Score:2)

            by delt0r ( 999393 )
            Your not going to be allowed to use it in volume outside a tank farm if its flammable. Fire is something you don't intend to happen but does anyway.

            • Re: (Score:2)

              by guruevi ( 827432 )

              As long as the oil remains contained within the unit, there is not a problem. Most oils aren't immediately flammable either. By the time most oils have the time and temperature to catch a flame, the sprinkler systems should have activated.

              • Re: (Score:2)

                by delt0r ( 999393 )
                Sprinkler system on flamable oil. You really have no idea about fire regulations do you.

                • Re: (Score:2)

                  by guruevi ( 827432 )

                  Water sprinklers in datacenters? You really have no idea about data centers do you?

                  • Re: (Score:2)

                    by delt0r ( 999393 )
                    Which is why its flammability will totally affect the permits on your ability to use the said oil dumbass. if it is not flammable and non toxic, such permits and handling procedures are easy and cheap. Which is why you want it to be non flammable! Back to the original post. Sheesh talking to wall here.

                    • Re: (Score:2)

                      by guruevi ( 827432 )

                      You seem to have failed basic physics and chemistry.

                      As I said:
                      Oil is not flammable, it's combustible, big difference.
                      No oxygen means no flame, regardless of the fuel
                      Datacenters remove the oxygen in case of fire, the fire suppression system uses an inert gas to displace oxygen.

                      You do not need permits to handle oils in these situations. You are not 'storing' oil.

                    • Re: (Score:2)

                      by delt0r ( 999393 )
                      Flammable Combustible? Really? I think your brain fell out. Good luck explaining that after the fact when they find you didn't comply with the legal requirements.
        • 9. Low dielectric constant. No need to slow down signals.
      • Sort of, not really an oil. It's a drivitive of Fluorinert which was developed by 3M and used extensively on Cray systems. The problem remains, once you have component failure you impart carcinogens into the fluid making it highly toxic. The other problem with this stuff is the price. Originally priced around $2K a litre (systems usually used hundreds of gallons), and now I've heard pricing down to $150/litre and some times cheaper. In any case it's neat looking stuff when you get to watch it boil away http [unixgr.com]
    • We are quite adept at job-specific innovation, especially that of insulation from the solvent-like characteristics of the oil(s) used as liquid refrigerant.

      Breaking the Computer Room Air Conditioning habit is a cute meme, but the transfer of heat through liquid refrigerant versus air is much more efficient.

      Think of the difference in your after party, immersing your hand in 232 Celsius versus 100 Celsius water.

    • Re: (Score:1)

      by catsRus ( 548036 )
      Electrical Line transformers have been oil cooled for a long time, also dummy loads for RF testing. Lots of prior art for cooling electrics and no damage to packaging etc.
    • > Would not oil start dissolving all the parts

      Have you seen the depreciation rate on supercomputers? If it's a race between the market price and the oil, my money's on the market.

  • The Cray-2 [wikipedia.org] was a much earlier large-scale use of computational elements immersed in inert cooling liquid.

  • Safety Hazard... (Score:4, Informative)

    by HockeyPuck ( 141947 ) on Monday July 06, 2015 @10:07PM (#50059997)

    I worked with a company that did a proof of concept for these "oil bath systems" It was just like in the photo, a rack on its' back immersed in a tank of oil.

    From a cooling standpoint, it was great, from a parts replacement, it's a disaster waiting to happen. It's not how hard it is to replace the part. We installed a small crane above the rack to make it easier to lift the server up out of the bath, so we could lay it down and replace mem/cards/SSD etc. A 1RU server can weigh anywhere between 30-40lbs, and a 2RU server can weigh almost 70lbs.

    We had to surround the bath will a perforated rubber mat, as oil+typical datacenter floor is a huge slip hazard.

    You've also got to install pumps and a heat-exchanger. Reminds me of the old water cooled mainframes. Just with way more plumbing.

    • The stuff they are probably using for this is the 3m engineered fluids that have a really high vapor pressure so the mess should just evaporate, and it's really low viscosity so the heat moves away from the parts by convection so no need for pumps, the heat exchanger is put above the fluid where it recondenses and falls back into the bath...
  • IBM 7302 Core Memory Unit [wikipedia.org] circa 1957-1958 for the IBM 7030 (Stretch) [wikipedia.org] computer.

    The core memory in the IBM 7302 was heated/cooled to stabilize its operating characteristics. Early units immersed the core stack in heated/cooled oil, later units called the IBM 7302A, blew heated/cooled air through the core stack.

    I once herd a story from a real old timer about fixing these memories. When they were new there was a problem with small metal particles left over from manufacturing floating around and shorting out

    • Heck, not even that long ago OC folks have been testing with the stuff for years. I recall reading several articles years ago, where people were taking souped up (pardon pun) PC's and immersing them in a liquid solution to test for cooling... They may or may not be using more less the same inert liquid.

      As I recall, while there were some benefits, there were significant problems as well. Maintenance was obliviously one, both of the components, but also the bath. Also it should be pointed out, while this does

  • Didn't Cray and others do this decades ago?

Slashdot Top Deals

Fast, cheap, good: pick two.

Close