Why AI and HPC are changing the way data centers think about cooling
Coming from racks operating around 10 kW, the industry is moving toward rack power densities of 1 MW and beyond. This represents a 100× increase in rack power in less than a decade and a fundamentally different cooling challenge.
AI and high-performance computing are changing the physical requirements of modern data centers. As computing becomes increasingly concentrated in high-density systems, power and thermal requirements are becoming more closely connected to the overall design of the facility. ASHRAE notes that the shift from CPU-centric to GPU-centric computing is driving new power densities, thermal loads and infrastructure requirements, making integrated power and cooling design increasingly important.
Increasing Compute Density
The shift toward GPU-intensive computing is creating significantly higher-density environments than many traditional data center deployments.
For example, NVIDIA’s GB200 NVL72 is a rack-scale, liquid-cooled system containing 72 Blackwell GPUs and 36 Grace CPUs. NVIDIA describes the system as a rack-scale liquid-cooled architecture designed for high-performance AI workloads.
But the progression is already moving beyond the current generation. Schneider Electric’s reference design for NVIDIA Vera Rubin NVL72 supports 188 kW per rack at MaxQ and 227 kW at MaxP. Looking further ahead, Rubin Ultra systems using the new Kyber rack architecture can reach 1–1.2 MW of power density per rack, with up to 576 GPUs in a rack.
At these densities, thermal management becomes more than a cooling-system consideration. It increasingly influences rack architecture, power distribution, facility design and future expansion.
The Move Toward Liquid Cooling
Air cooling remains an important part of data center infrastructure, particularly for lower-density workloads. However, as heat density increases, liquid cooling is becoming an increasingly important tool for managing high-performance computing environments.
ASHRAE identifies direct-to-chip liquid cooling and rear-door heat exchangers among the architectures being used to support high-density AI infrastructure. Its framework notes that purpose-built AI data centers can routinely exceed 50–120 kW per rack, with densities expected to trend higher.
This does not mean there is one universal cooling solution. Different workloads and facilities require different approaches.
Where Immersion Cooling Fits
Immersion cooling offers a fundamentally different approach by placing compatible IT equipment directly into a dielectric cooling fluid. Rather than relying primarily on air to transport heat away from components, the surrounding fluid provides direct thermal contact with the equipment.
As rack power densities continue to increase, this creates opportunities for efficient heat removal and alternative approaches to facility-level thermal management.
The challenge now is moving beyond technical potential toward broader deployment — including equipment compatibility, operational practices, infrastructure integration, standards and supply-chain readiness.
From Cooling Technology to Infrastructure Strategy
As AI infrastructure continues to evolve, cooling decisions increasingly need to be made alongside decisions about power, compute architecture and facility design. The progression toward hundreds of kilowatts and ultimately megawatt-class racks means that these systems can no longer be considered independently.
This creates an opportunity for greater collaboration across industries. Experience from automotive, energy storage, data centers and other high-density thermal applications can contribute to how the industry approaches the next generation of computing infrastructure.
The future of AI infrastructure will not be determined by compute alone. It will also depend on the infrastructure capable of powering, cooling and scaling it.
