Is immersion cooling ready for mainstream?

As AI and high-performance computing push the boundaries of heat generation, traditional air cooling methods are no longer viable. For the longest time, it was easy to cool a server CPU with a heatsink and a fan, but with power draws for CPUs hitting 400 watts and GPUs hitting 700 watts, that combination is no longer enough.

Liquid heat cooling is a viable alternative. Liquid has a higher heat absorption rate than air, so it can handle much hotter loads. The two most common methods of liquid cooling are direct-to-chip cooling and rear door heat exchange. Both use liquid to draw away heat, but at no point does the equipment actually get wet.

With immersion cooling, however, hardware is getting soaked like a dirty dish.

What is immersion cooling?

Unlike the other two methods, immersion actually involves dunking very expensive electronics in a liquid bath, but it is a non-conductive liquid, so the equipment is not fried. Immersion uses dielectric liquids, which are non-conductive, so they are safe to use with electronics. Examples of dielectric liquids are oils, such as mineral oil or petroleum oil.

Immersion has a number of advantages over traditional liquid cooling. For starters, no pumps are needed. The hardware is sitting in liquid instead of being pumped through tubes. This lowers the electric bill. Secondly, because the liquid is in direct contact with the electronics, it can be more efficient and effective at cooling.

Because of this, immersion has an astonishing power usage efficiency (PUE) rating. PUE is a measure of the amount of electricity needed to cool IT equipment. A PUE of 2.0 means that for every dollar spent to power a server, another dollar is spent to cool it. A PUE of 1.5 means that for every dollar to power the server, it costs fifty cents to cool it.

The ultimate goal is to get as close to 1.0 as possible. According to the Uptime Institute, the average data center PUEs is 1.5. With liquid cooling, it can get as low as 1.1. In one astonishing case, a Hong Kong data center got to a PUE of 1.01, meaning it cost next to nothing to cool it.

Types of immersion cooling

There are two main types of immersion cooling: single-phase and dual-phase or two-phase.

Single-phase immersion cooling is the simplest type. It involves submerging electronic components in a dielectric liquid. The heat generated by the components is transferred directly to the liquid, which is then cooled by a heat exchanger. In a single-phase environment, the tanks are open and the hardware is readily accessible.

Dual-phase immersion cooling is more complex. It involves submerging electronic components in a dielectric liquid that has a lower boiling point than the fluid used in single phase cooling. When the liquid boils, it turns into a vapor, which rises inside the tank. In dual phase cooling, the tanks are closed. The vapor is then cooled and condensed, falling back into the liquid.

When to use immersion cooling

IT now has several cooling options for data center equipment; air cooled, liquid cooled, and immersion. So how do you differentiate and decide which to use? The calculus is complex.

When it comes to air cooling, the decision is made for you by rack density. Cooling with the traditional heatsink and fan method works when rack densities are at or about 20 kW per rack, which is suitable for the vast majority of workloads. A database server or a file server is not going to run particularly hot.

At more than 30kW, air cooling is no longer able to absorb all the heat. That gets into the realm of compute intensive loads, like HPC and AI, with the processes running at full utilization for a sustained period.

“That’s probably the biggest reason why you’re seeing so much interest in liquid cooling, and that’s certainly now being driven by AI,” said Joe Capes, CEO of immersion cooling vendor LiquidStack. “I think the other aspect of it is just more around ESG and sustainability because as a planet we have finite resources.”

Air cooling uses immense amounts of water for air conditioning. Capes said that data centers are one of the largest users of water for refrigeration, with data center consuming 660 billion liters of water in the United States per year for cooling data centers. “I’ve heard it said that every ChatGPT request uses a bottle of water,” Capes said.

One of the Achilles heels of the direct-to-chip method is you need air cooling, said Capes. And with every hardware refresh, you have to re-engineer and redesign the direct-to-chip system at the board level, and which means replacing about half of the capex with every refresh.

“Immersion certainly lends itself more easily to new builds. I’m not saying it’s exclusive to new builds, but it certainly lends itself to new builds and modular and that’s because you can virtually eliminate air cooling all the infrastructure around that, and it makes the economics fairly, highly favorable,” Capes says.

The downside to immersion

But there are also drawbacks, not the least of which is getting the data center ready for the tanks. “I would have to think that preparing the data center for these tanks has to be a major undertaking,” said Abhijit Sunil, senior analyst with Forrester Research. He added that the immersion cooling is not something you can just add on to an existing data center infrastructure. “I think immersion cooling is better to consider on net new builds or retrofits.”

Sean Graham, senior analyst with IDC, said that a retrofit is required on any kind of liquid cooling. “That’s why there’s a short term popularity in rear door heat exchangers, because you can cool some of those higher densities without gutting the whole data center. Rear door cooling simply attaches directly to the back of the existing rack, you don’t need a redesign, you just need to be able to hook them up to the cooling agent,” he said.

Another problem is the space consumed, Sunil said. Tanks can be stacked as high as a standard datacenter rack and since their size varies from one vendor to the next, you may lose floorspace and wind up with lower capacity than you had expected.

Graham concurs. “I was in a data center that had liquid immersion cooling, and they lost about 50% of their floor space to cooling tanks,” he said.

For that reason, both recommend judicious use of immersion cooling for only the extreme conditions of high heat density, which is not going to be the entire data center, only a portion of it.

Adding to the challenge is that immersion is being done with hardware that was never meant to be dunked in liquids. “I think that probably one of the biggest challenges in the market up until now has been getting hardware that is immersion-optimized and warrantied,” said Capes.

The immersion cooling process involves dunking motherboards into the tanks of liquid, not a full chassis. HPE, Dell Technologies, and Lenovo don’t sell motherboards, they sell a rack enclosure. Ty Schmitt, vice president and fellow at Dell, says the company has an OEM business unit that works directly with its immersion cooling partners to take existing Dell servers and do what is needed to make the hardware work in an immersion system.

“From a validation qualification and warranty standpoint, it allows the guts of that system to be able to operate in said immersion cooling tank or technology. So, we have a dedicated team that owns that and does that on behalf of our partners and customers,” he said.

Immersion cooling growth potential

The market for immersion cooling is still in its infancy. Grand View Research puts the global immersion cooling market size at $197 million in 2022, anticipated to grow at a compound annual growth rate (CAGR) of 22.6% from 2023 to 2030.

Maurizio Frizziero, director of cooling innovation and strategy at Schneider Electric, said the real challenge is not technological, but cultural. IT managers just need to wrap their heads around the idea that they are going to dunk motherboards in liquid to cool them and not fry them.

“We believe that liquid cooling is a very viable way to go in a very sustainable way. Of course, we need to win the concern that any operator has on a new technology or new design guidelines,” he said, adding that Schneider sees immersion cooling and liquid cooling in general growing at a double-digit CAGR in the next three years.

Schmidt says he has noticed increasing interest from Dell customers in immersion as a solution to power constraints, since immersion uses the least amount of power of the cooling systems available.

“We understand what their power profile is and where it’s going to be in the future and what their facility constraints are. Immersion cooling may be a great solution for them. So, we have to understand both the customer constraints and environment as well as what’s happening in the industry to help guide that given customer, and the answer varies from customer to customer,” he said.

The big data center providers like Equinix and Digital Reality Trust are only now getting religion on immersion, said Capes. “We hadn’t gotten much support up until last six months. I can tell you that like the big guys are road-mapping immersion-ready servers on a two-year horizon. So, you can fully expect that within the next two years, you’re going to see the big players with air-cooled systems, direct-to-chip cooled systems, and immersion all side by side, and you’ll be able to pick and choose what you want,” he said.