Liquid Blade Brings Immersion Cooling To Blade Servers 79
1sockchuck writes "In the past year we've seen several new cooling systems that submerge rack-mount servers. Now liquid immersion cooling is coming to blade servers. Liquid-cooled PC specialist Hardcore Computer has entered the data center market with Liquid Blade, which features two Intel 5600 Xeon processors with an S5500HV server board in a chassis filled with dielectric fluid. Hardcore, which is marketing the product for render farms, says it eliminates the need for rack-level fans and room-level air conditioning. In recent months Iceotope and Green Revolution Cooling have each introduced liquid cooling for rack-mount servers."
I want these at my data center... (Score:5, Funny)
Re: (Score:3, Funny)
Hell, I'd get promoted if I did that... maybe you need to get your boss's job?
Or you would get a raise. (Score:1)
Re: (Score:2, Insightful)
So you have to pay twice as much?
Re: (Score:2, Interesting)
my server is leaking (Score:4, Funny)
Re:my server is leaking (Score:4, Funny)
Do you need a snorkel and flippers to swap a network cable?
Upholding Moore (Score:4, Interesting)
Do we really NEED liquid cooled servers in datacenters? Is this just our feeble attempt to validate Moore's Law despite diminishing returns on smaller process size and core multiplication...?
What the hell am I talking about? Of COURSE we need them!
Re:Upholding Moore (Score:4, Insightful)
Do we really NEED liquid cooled servers in datacenters? Is this just our feeble attempt to validate Moore's Law despite diminishing returns on smaller process size and core multiplication...?
Yes. No.
The massive densities you can achieve with liquid-to-liquid cooling allows for much smaller data centers (or much more performance in existing data centers).
Just being able to build a smaller data center can mean you've recouped the liquid cooling investment, even before factoring in the savings for increased cooling efficiency/watt, no AC, and no cooling fans.
serviceability (Score:5, Interesting)
How hard is it to say; change a disk in one of the submerged nodes ? or fix a loose ethernet cable ? If the nodes are separated in compartments, and you could isolate and drain one while servicing it, this would be really nice indeed.
Re:serviceability (Score:5, Funny)
Re: (Score:2)
It sounds neat until you hear about how all the hair off Bob's ass accumulated and blocked the exhaust port and you need to get in there right now and remove the hair before the server farm explodes.
In other words.. a suicide mission.
Re: (Score:2)
I didn't know you could SCUBA dive naked.....
Re: (Score:1)
Re: (Score:3, Informative)
that's PADI: http://www.padi.com/scuba/ [padi.com]
unless you were thinking about rice.. ;hot water, rice... server farms finally derserve their names !
Re: (Score:1, Interesting)
you could connect disks with esata?
Re: (Score:1)
I'd guess SSD and network attached storage.
Re:serviceability (Score:5, Funny)
How hard is it to say; change a disk in one of the submerged nodes ? or fix a loose ethernet cable ? If the nodes are separated in compartments, and you could isolate and drain one while servicing it, this would be really nice indeed.
Did you see the movie Sunshine [imdb.com]? You'll have to immerse yourself in the coolant, possibly freezing and/or bleeding to death after getting your leg stuck in the rack. It had better be an important upgrade.
Re:serviceability (Score:5, Interesting)
That's exactly what got my attention. In the article, the CTO of hardcore computer is quoted as saying that "Our Core Coolant is 1,350 times better than air, by volume.". I don't know how that works out in energy spending when compared with air but if it has a linear relationship with the energy cost of cooling, I really doubt if the hypothetical energy savings can bring a net positive when considering the additional cost associated with meddling with the hardware, whether by maintenance or by hardware upgrades. After all, this slashvertisement is oh so keen in lauding the qualitative and subjective advantages of this toy but it doesn't even come near presenting the costs associated with forcing your company to be profoundly dependent (if not held hostage) of hardcore computer for support, maintenance of both hardware and cooling rig and upgrades.
Re: (Score:3, Interesting)
it's mainly bullshit. my guess is, he's talking about the ability of the liquid to retain heat, which is nice and all, but you've got to get that heat in, and then out. basically, you're adding one motor (thing still needs a fan somewhere, plus the water pump), lots of tubing, liquid... i wouldn't use that in my home PC, let alone in a server room.
Re: (Score:2)
Re: (Score:2)
After all, this slashvertisement is oh so keen in lauding the qualitative and subjective advantages of this toy but it doesn't even come near presenting the costs associated with forcing your company to be profoundly dependent (if not held hostage) of hardcore computer for support, maintenance of both hardware and cooling rig and upgrades.
how is this different to choosing a blade solution from another OEM? That is why OEM's love blades because you are generally locked into there hardware for the next 4-7 years due to the life of the chassis lasting that long.
Re: (Score:3, Informative)
How hard is it to say; change a disk in one of the submerged nodes ? or fix a loose ethernet cable ? If the nodes are separated in compartments, and you could isolate and drain one while servicing it, this would be really nice indeed.
Have you ever been in a huge data centre? There are mostly systems not needing any human interaction for years and in fact people need a map and an index to find the system. Human intervention -e.g. for failing hardware- is usually cast into procedures whereby an "operator" does the work needed. Disk storage is usually separated form application servers. Etc...
Huge data centres analyse the used hardware used and usually set up a palette of system types to select from. (Therefore anything outside of "the
Re:serviceability (Score:5, Informative)
We're talking about blade servers. they're not submerged nodes. they're submerged blades. storage happens on a SAN. What fucking year is it, anyway? In this design (big fat picture in the TFA, you lazy, reactionary fuck) each blade is sealed into its own unit which can be pulled separately. So it's even more of a non-issue. You just want something to complain about, when there is nothing to complain about. Thanks for helping make slashdot grate.
Re: (Score:1)
We're talking about blade servers. they're not submerged nodes. they're submerged blades. storage happens on a SAN. What fucking year is it, anyway? In this design (big fat picture in the TFA, you lazy, reactionary fuck) each blade is sealed into its own unit which can be pulled separately. So it's even more of a non-issue. You just want something to complain about, when there is nothing to complain about. Thanks for helping make slashdot grate.
Wow, nice people skills. Name calling is so cool.
Re: (Score:2)
Wow, nice people skills. Name calling is so cool.
I'm not here to 'keep it real, dawg'. I'm here to say what I have to say in a manner that is pleasing to me, and slay any trolls I meet in the process. Dawn take you all, and be stone to you!
Re: (Score:2)
Uuum, and you did expect what from a troll who calls himself “drinkypoo” (and also does that in RL)?
He is an elaborate troll though, as he has several sock puppet accounts who regularly get mod points, which he uses for the real trolling: Moderating things in his (trollish) favor.
Even trolls hate him, as you can see when he clashes with other trolls. He’s a subhuman of the worst kind. And I don’t mean this as an insult, but as an objective measurement.
Re: (Score:3, Insightful)
I would not characterize what he said as reactionary. He does have a valid question, which is how easy is this to service? Your right, that the hard drive is a non-issue since you would want to use SAN, but hard drives generate heat too, so why would we not want it for that too?
Not everybody uses Blades. I looked into it and I found it costly and proprietary compared to other solutions that could provide even greater density.
Even if we did create a completely sealed 1U server case, we would still need to
Re: (Score:2)
I would not characterize what he said as reactionary. He does have a valid question, which is how easy is this to service?
One which would have been answered by loading RTFA (I know, I know) and looking at the big fat picture at its head, and seeing that blades are in enclosed modules containing coolant.
Your right, that the hard drive is a non-issue since you would want to use SAN, but hard drives generate heat too, so why would we not want it for that too?
You can't immerse hard drives and you can't create immersible hard drives without creating whole new classes of issues. Conner tried making non-vented disks back in the early days of ATA and look where it got them. You could have a pressure-regulating mechanism like the nitrogen bladder in typical shock absorbers (what a terribl
Re: (Score:2)
I don't think that answers the question at all. Unless you are saying that the Blade itself is completely sealed and non-serviceable. I don't know if that is true, I would imagine some parts on a Blade are
Re: (Score:2)
I don't see why the topic of liquid cooling hard drives isn't on the table. SAN or no SAN hard drives generate heat regardless of where they are housed. The individual drives can be liquid cooled even if not directly submerged in liquid.
http://www.frozencpu.com/products/7500/ex-blc-487/Koolance_HD-60_Hard_Drive_Liquid_Cooling_Block.html [frozencpu.com]
Re: (Score:2)
Hard drives cannot be directly submerged in liquid. However, I feel that it should be possible to design a high density liquid cooled SAN. As you can see in the Koolance product even a block a few mm's wide should be enough to do it. So creating a block combined with a SATA backplane holding many hardrives close together should work out just fine.
Re: (Score:2)
RTFA.
And I like how they are using an oil like mineral oil. Mineral oil is a great insulator, its used in large transformers for cooling and insulating the windings. It is also used in high voltage oil circuit breakers for arc quenching.
I once thought of using it to submerge PC hardware for cooling but the idea of oil all over my components was not very attractive. If I had to swap something out or upgrade/add I would have to move the PC to the bath tub or kitchen to remove the component and clean the oil o
Liquid blades? Sounds familiar... (Score:1)
Re: (Score:1)
My guess would have been Terminator 2.
Re: (Score:1)
And if the dielectric fluid is non-compressible... (Score:2, Funny)
I'd like to be under the sea in an octopuses data garden in the shade...
Re: (Score:2)
Re: (Score:1)
Just friendly crustaceans
The blind leading the ignorant. (Score:2, Insightful)
Immersion liquid cooling is something I have done in the past, and that is all well and good, it is after all HOBBY level tech.
For commercial level tech it isn't even a joke, imagine opening the bonnet / hood of your new 2010 car and finding a big tub full of water with the engine immersed in it.
Internal combustion engines have had closed circuit internal liquid cooling circuits for decades, and frankly computers and electronics have had closed circuit internal liquid cooling circuits for decades too.
Think
Re: (Score:1)
I'm a bit out of date perhaps (I read about this a few years back) but what is the regular kind that you refer?
Re: (Score:2)
Here are two real world examples.
1/ the traditional electric kettle, normally a 3 kW element immersed in a few pints of water, or the traditional hot water cistern, a 6 kW element immersed in a few tens of gallons of water.
The above works on open circuit (immersion) and thermosyphon / convection.
2/ the traditional electric showed, normally a 8 kW element in a unit a tad smaller than a soda can.
The above is closed circuit and forced flow.
Simple changing from immersion and convection, to closed circuit and fo
Re: (Score:2)
damn, that should be "the traditional electric SHOWER"
Re:The blind leading the ignorant. (Score:4, Informative)
Immersion liquid cooling is ... all well and good, it is after all HOBBY level tech.
Really? Cray started doing this back in 1985 [wikipedia.org], so I wouldn't call it "HOBBY level tech."
Re: (Score:2)
"1.5GBytes of DDR 2 RAM"
HAHAHAHAHA ... no.
Re: (Score:1)
*Sigh*
Lets assume, for a second, that I have a system that scales linearly from 12 to 48 cores (2S Xeon 56xx/Opteron 2xxx to 4S Opteron 6xxx) and requires ~4GB per core. (48-192GB)
Now, lets assume that I'm willing to switch to ARM.
1. I will need a board that with at-least 24-96 soon-to-be-released ARM 2.0 Ghz Coretex A9 dual core core CPUs.
2. This board will also need unbelievably wide (and extremely complicated) crossbar, as I can no longer simply put 4 independent buses (one per CPU) and simply connect th
What's wrong with tube liquid cooling? (Score:1)
I don't know why you'd have to immerse the entire blades in cooling.
Couldn't you just use tube liquid cooling, found in many enthusiast machines? You could make one pump/heat exchanger per blade enclosure, with a custom valve/fitting connecting the blade to the enclosure to pass the cooling around. I'm sure a clever engineer could easily come up with a design for redundant heat exchangers/pumps in the enclosure, even hot swappable.
The benefits of liquid cooling (low noise, lower temperatures) with less or
Liquid Blade? (Score:2)
Is that similar to a Light Saber?
Re: (Score:1)
Re: (Score:2)
It doesn't *have* to run Ubuntu.
Renderfarms will not buy this (Score:1, Informative)
> marketing the product for render farms
I used to have root on a renderfarm with a few thousand cores and this is exactly the type of fiddly tech you want to avoid when you scale-out. Many renderfarms are located at multiple sites, mainly due to legacy but also for convenience and resilience. This means the majority of a renderfarm may be co-located where land and electricity is cheap and rely on high bandwidth connections to offices where transport and talent is plentiful. I know of two renderfarms whic
Re: (Score:1)
Well, it might be useful for new render farms. They could save money by not getting heavy cooling in the first place, eh?
However, the technology is radical and yet to be proven yet. I don't see any business adopting or trying this, until it has been proven -- or until it becomes inexpensive to try.
It's not worth paying twice as much for a server, for some fancy immersive cooling technology, if it hasn't yet been proven for large deployments.
I think it's more of an option (immediately) for small-sca
liquid heatsinks? (Score:2)
Does anyone make a liquid cpu heatsink, something to slap over your cpu (and seal)? Seems like that would be a nice innovation if it could be kept sealed.
Re: (Score:3, Informative)
I doubt sealing to the board would be very practical, it would be very hard to get a good enough seal there and anyway most of the heat comes out of a CPU through direct conduction to the heatsink anyway so I don't see a whole lot of point in immersing the CPU itself.
What you can get easily are "waterblocks" which attatch in place of a regular heatsink and take the heat from the CPU in the regular manner but are designed to transfer it to piped water rather than to the air.
Re: (Score:2)
Past is prologue (Score:2)
Cray Supercomputers were doing this a quarter of a century ago. Fluorinert to the rescue!
this will cost a mint with real deployment (Score:1)
Immersion cooling makes sense from a lot of perspectives. However, there is one enormous problem. That is that chips aren't rated to work when immersed. You will have to get the companies who make the chips to specify that the packages the chips are in will are designed and safe to use in liquid. And they're going to charge you a lot of money for that. Frankly, it'll make the device non-cost effective.
You can just omit this step, and throw something together with air-rated chips and hope it works anyway. Bu
Remember these rules when using liquid cooling (Score:2)
For special and expensive systems such ideas may be useful, but it is hard to beat air for commodity systems because it is free, there is plenty of it, it cannot spill, it does not put excess load on the floor, it is inert (to electronics), and it can be easily recycled.
Re: (Score:1)
No, but they throw in a free Rubbermaid bucket and a 5-pack of ShamWows with every rack purchased.
Re: (Score:1)
The fluid is dialectric and therefore non-conductive and 'safe' it won't short out the other servers.
However, it could create a mess. If you have 1 full immersion blade you'll want them all that way.
CPU and all components are immersed in the fluid. The primary danger behind 'leaking' is that the tank runs low and the server or pump starts overheating due to lack of fluid.
Re: (Score:1)
The fluid used in immersive cooling systems IS dialectric (an insulator) and inert to electronics.
Otherwise, it would make no sense to immerse the blade's components such as CPU and memory in it during normal operation.
This is not a water cooling solution, or a water block / partial immersion solution, where a dangerous liquid might be used, but kept isolated from the components by using tubing, a cooling block, and a pump.
In an immersion solution like the one discussed, all components of the server
hm.. perhaps a cheaper way to reduce vibrations? (Score:1)
Than re-fitting all your racks with some carbon fiber advanced material. Surely cooling with a pumped liquid generates less vibration in the air than having a bunch of high-speed fans in your case.
Maybe this will result in better hard drive performance :)
I wonder (Score:1)
Water is dielectric by itself.. Electric conduction is because of all the mixed salts in it. Perhaps they are just selling distilled water? :)
Anyway, you would have to change the liquid from time to time, because some dust or anything might get into it and dissolve, thus making it conductive..
Liquid blader T1000 ... (Score:1)
... killing machine comes handy both at home and at the office. Call now, and order directly from the creator Cyberdine corporation.
Good luck with that! (Score:2)
That “dielectric fluid” is really nasty stuff. To give you an idea, how nasty it is, I have an example for you:
Say you have attached your mouse to the computer, and sealed everything off with hot glue and everything. Now you fill the thing with that oil.
Then that stuff creeps trough the connector, trough the inside of the cable (between the wire and the plastic), up to your mouse, out your mouse, over all your table and everything, down to your floor and trough the whole room. Unless you stop it