Abstract Draft

AI datacenters are rapidly moving toward warm-water direct-to-chip liquid cooling as rack power densities exceed the limits of air cooling. However, the main research frontier is no longer heat removal alone; it is the dynamic co-control of AI workloads, GPU power states, coolant flow, coolant temperature, CDUs, and facility heat rejection. This paper proposes that workload-aware direct-to-chip co-control can improve energy efficiency, water use, and thermal reliability compared with fixed coolant settings, but only when optimized against transient rack-to-facility dynamics rather than steady-state chip temperature alone. The paper will defend this hypothesis using evidence from hyperscale infrastructure disclosures, standards work, emerging coordinated cooling-compute research, and dynamic liquid-cooling studies. It then proposes an open benchmark for liquid-cooled AI racks that combines workload traces, thermal models, rack/CDU behavior, facility-water conditions, and whole-system metrics such as throughput per joule, cooling energy, pump energy, thermal cycling, water use, and throttling risk.

Core Thesis

AI datacenter cooling is moving from a heat-removal problem to a dynamic control problem. Warm-water direct-to-chip liquid cooling is becoming the practical path for dense AI racks, but the industry still lacks open evidence showing how these systems should be controlled during real, changing AI workloads. This paper argues that the next high-value research frontier is workload-aware direct-to-chip co-control: coordinating AI workload behavior, GPU power states, coolant flow, coolant temperature, CDU response, and facility heat rejection against transient rack-to-facility dynamics.

The argument is simple: if AI workloads change quickly, cooling systems should not be controlled as if the heat load is steady. A rack running bursty inference, synchronized training, power-capped jobs, or mixed workloads produces changing heat patterns that move through chips, cold plates, manifolds, CDUs, facility-water loops, and outdoor heat rejection systems. A cooling strategy that only targets steady-state chip temperature can miss energy savings, water savings, thermal reliability risks, and compute-performance tradeoffs.

Testable Hypothesis

A workload-aware warm-water direct-to-chip co-control strategy can improve AI datacenter efficiency and thermal reliability compared with fixed coolant-flow and fixed supply-temperature control, but only if it is optimized against transient rack-to-facility dynamics rather than steady-state chip temperature alone.

Why This Hypothesis Matters

Modern AI infrastructure is becoming dense enough that cooling is part of the compute architecture. Google has discussed racks scaling from 100 kW toward 1 MW and has described liquid cooling as essential because chip power has moved from roughly 100 W devices toward accelerators above 1000 W (Google Cloud). NVIDIA’s Rubin-era architecture points toward fully liquid-cooled rack systems using warm liquid loops, including 45 C coolant operation and dry-cooler-based heat rejection in favorable climates (NVIDIA Blog). OCP has also identified CDUs, telemetry, control interfaces, and AI racks exceeding 500 kW as areas needing collaborative standards (Open Compute Project).

These sources show that the main deployment question is no longer whether liquid cooling is needed. The harder question is how liquid-cooled AI infrastructure should be operated as a whole system. The paper will defend the claim that cooling control must move closer to workload behavior, because AI workload dynamics shape heat generation before the cooling system sees the heat.

Background and Motivation

Direct-to-chip liquid cooling removes heat close to the processor by circulating coolant through cold plates attached to GPUs and CPUs. This is attractive because liquid carries far more heat than air, allows denser racks, and can reduce fan and mechanical cooling energy. Schneider Electric describes direct-to-chip systems as closed-loop rack-level systems where cold plates, manifolds, quick disconnects, and CDUs work together, with heat ultimately rejected through facility water infrastructure (Schneider Electric).

Warm-water direct-to-chip cooling is especially important because higher coolant temperatures can make outdoor dry-cooler operation easier and reduce the need for chilled water. NVIDIA argues that 45 C liquid cooling can reduce mechanical cooling requirements and, in some climates, nearly eliminate evaporative water use by rejecting heat through closed-loop dry coolers (NVIDIA Blog). This changes the research problem: the goal is not just to keep chips cold, but to keep chips safe while using the warmest practical coolant, the least pump power, the least water, and the most useful heat-rejection path.

The missing piece is transient coordination. AI datacenter cooling is often discussed with peak rack power, average rack power, coolant inlet temperature, and chip temperature. Those are necessary metrics, but they do not fully describe what happens during workload changes. If an inference burst or training phase suddenly changes GPU power, heat first appears at the chip, then moves through the package, cold plate, coolant, rack loop, CDU, facility loop, and outdoor heat rejection equipment. Each layer has delay, inertia, limits, and control choices.

Existing Evidence

The strongest direct evidence for the paper’s direction is the 2026 paper “Coordinated Cooling and Compute Management for AI Datacenters.” It proposes a hierarchical framework that co-optimizes GPU parallelism, DVFS, and cooling control knobs using real Azure inference traces and GPU profiling (arXiv). This paper is important because it links AI serving decisions to thermal and cooling objectives rather than treating cooling as a fixed background cost.

Industry evidence points in the same direction, though it should be treated more carefully because some claims are vendor-reported. ProphetStor describes a predictive workload-aware liquid-cooling system that combines GPU telemetry, rack-level coolant data, and Kubernetes job metadata to forecast heat and adjust pump and valve setpoints (ProphetStor). Its claims are useful because they show what operators and vendors believe the control stack should become: a closed loop between workload intent, GPU telemetry, coolant flow, and CDU behavior.

There is also experimental work showing that dynamic cooling ideas can improve direct-liquid cooling. A direct-liquid-cooling reliability study describes regulating coolant flow to thermal test vehicles using flow control devices, supporting the idea that direct-to-chip cooling can be actively controlled rather than run at one conservative setpoint (NSF PAR). More recent dissertation work on hybrid-cooled servers and single-phase immersion systems frames liquid cooling as moving from isolated component improvements toward coordinated, energy-aware, scalable architectures for dynamic workloads (UTA MavMatrix).

Broader datacenter control literature also supports the proposal. Reviews of dynamic thermal environment management and data center cooling optimization show that sensing, prediction, control, and model predictive methods can reduce energy use when cooling is treated as a dynamic system. However, much of that work was built around air-side or facility-level control, not modern liquid-cooled AI racks with GPU workload traces, direct-to-chip cold plates, CDUs, and warm-water heat rejection.

Research Gap

The field lacks an open, validated benchmark for workload-aware direct-to-chip co-control under realistic AI workload dynamics. Existing sources tend to solve pieces of the problem:

• Hardware vendors show liquid-cooled rack architectures.
• Standards groups call for common interfaces, telemetry, and commissioning methods.
• Cooling papers test cold plates, manifolds, flow rates, or facility systems.
• AI systems papers optimize scheduling, serving latency, or GPU utilization.
• Emerging co-control papers connect compute and cooling, but public datasets and rack-to-facility liquid-cooling validation are still thin.

There is a missing benchmark layer between these pieces. A useful benchmark would connect workload traces, GPU power and temperature, chip/package thermal maps, coolant flow, cold-plate behavior, CDU response, facility-water temperature, heat rejection, cooling energy, water use, and reliability metrics. Without that layer, it is hard to know whether one cooling architecture or control method is truly better than another.

Proposed Research Questions

1. When do workload-aware cooling controls outperform fixed coolant-flow and fixed supply-temperature controls in liquid-cooled AI racks?
2. Which control variables matter most: workload scheduling, GPU DVFS, power capping, coolant flow, coolant supply temperature, CDU pump behavior, or facility-water temperature?
3. How much of the benefit comes from avoiding overcooling versus preventing thermal excursions and throttling?
4. Can higher warm-water supply temperatures improve full-site efficiency without increasing thermal cycling or reliability risk?
5. What open benchmark structure would let researchers compare single-phase direct-to-chip, two-phase direct-to-chip, hybrid air-liquid, and advanced cold-plate systems fairly?

Proposed Method

The paper proposes a benchmark-centered study built around a representative liquid-cooled AI rack model. The study can begin in simulation, but it should be designed for later validation on a thermal test vehicle or real rack.

System Boundary

The benchmark should include the full rack-to-facility chain:

• AI workload traces for training, inference, mixed workloads, and stress tests
• GPU power, utilization, temperature, and optional DVFS states
• Chip/package thermal maps or simplified thermal zones
• Cold-plate thermal resistance and pressure-drop curves
• Rack manifold behavior and coolant distribution
• CDU pump, heat exchanger, valve, and control response
• Facility-water inlet temperature and flow assumptions
• Outdoor dry-cooler or chiller operating assumptions
• Cooling energy, pump energy, fan energy, and water use
• Thermal reliability metrics such as peak temperature, thermal cycling, and time near limits

Baseline Control

The baseline should represent common conservative operation:

• Fixed coolant supply temperature
• Fixed or rule-based coolant flow
• Cooling sized for peak rack load
• Thermal constraints based mainly on chip or coolant temperature limits
• No direct awareness of upcoming workload behavior

Workload-Aware Co-Control

The proposed control strategy should use workload and telemetry signals to adjust both compute and cooling. Candidate control inputs include:

• GPU power capping or DVFS
• AI job placement and scheduling
• Tensor or request parallelism for inference
• Coolant flow rate
• Coolant supply temperature
• CDU pump speed
• Valve position
• Facility-water temperature setpoint
• Heat-reuse or dry-cooler dispatch

Candidate algorithms include model predictive control, hierarchical optimization, reinforcement learning with safety constraints, and simpler rule-based predictive control. A practical first version should favor model predictive control or hierarchical optimization, because the control problem has physical constraints, delays, and reliability boundaries.

Workload Cases

The benchmark should include several workload patterns:

1. Steady training: sustained high utilization with relatively predictable heat.
2. Bursty inference: short high-power bursts with idle or low-power valleys.
3. Mixed training and inference: competing latency and throughput requirements.
4. Power-capped operation: constrained electrical capacity or grid-event response.
5. Thermal stress case: elevated ambient temperature or high facility-water inlet temperature.
6. Fault or degradation case: partial flow restriction, pump response limit, or sensor drift.

Evaluation Metrics

The paper should argue for whole-system metrics:

• Tokens per joule or training throughput per joule
• Cooling energy per unit of AI work
• Pump and CDU energy
• Peak junction or case temperature
• Time above thermal warning threshold
• Thermal cycling amplitude and frequency
• Throttling events
• Water use effectiveness or avoided evaporative water use
• Heat-reuse value
• Reliability margin under degraded operation

Expected Results

The hypothesis predicts that workload-aware co-control will show the largest benefit when workloads are dynamic. Bursty inference should be especially favorable because fixed-flow cooling is likely to waste pump energy during valleys while still needing fast response during bursts. Mixed workloads should also benefit because some jobs can be shifted, delayed, or power-shaped without violating user-visible performance constraints.

The gains may be smaller during steady training. If a rack runs at nearly constant high utilization, the value of prediction and scheduling may be limited because there are fewer valleys to exploit. This is important because the paper should not suggest that smarter control always wins, but that smarter control wins when workload transients, thermal inertia, and facility constraints create exploitable flexibility.

The expected technical result is not simply a lower chip temperature. In fact, the best system may run chips safely but warmer, because warmer coolant can reduce chiller use, enable dry-cooler operation, reduce water consumption, and increase heat-reuse value. The better target is controlled thermal risk per unit of useful AI work.

Counterarguments

Counterargument 1: Fixed liquid cooling is good enough.

Some operators may prefer fixed or conservative controls because they are easier to certify, operate, and maintain. The response is that fixed control may be acceptable for early deployments, but it can hide large energy, pump-life, and water-use penalties at scale. As racks approach hundreds of kilowatts or megawatt scale, small inefficiencies become large operational costs.

Counterargument 2: Every datacenter is too different for a common benchmark.

Datacenters differ by climate, facility water design, rack density, electrical architecture, and operator priorities. The response is that benchmarks do not need to represent every site perfectly. They need to provide common reference cases, so researchers can compare methods under shared assumptions and then adapt them to real facilities.

Counterargument 3: Workload traces and hardware details are proprietary.

This is a real barrier. Hyperscalers may not publish detailed GPU traces, chip maps, or CDU control data. The answer is to create a tiered benchmark: open synthetic traces and thermal models for public comparison, with optional private calibration for operators that have proprietary data.

Counterargument 4: Co-control may increase operational risk.

Dynamic control can create new failure modes if it changes pump speeds, valve positions, or GPU power too aggressively. This is why the proposal emphasizes safety constraints, slew-rate limits, degraded-operation cases, and reliability metrics. A successful co-control strategy must improve efficiency without creating unstable thermal behavior.

Expected Contribution

This paper would contribute three things.

First, it would clarify the research gap: AI datacenter cooling needs transient rack-to-facility benchmarks, not just better cold-plate tests. Second, it would propose a testable hypothesis connecting workload-aware control to energy, water, compute throughput, and reliability. Third, it would define a benchmark architecture that can be used by thermal engineers, AI systems researchers, datacenter operators, and standards groups.

The practical contribution is that it gives operators a better way to ask procurement and design questions. Instead of asking only “Can this system remove enough heat?”, they can ask “How does this system behave during real AI workload changes, and how should it be controlled to optimize energy, water, performance, and reliability together?”

Proposed Paper Structure

1. Introduction

Introduce the shift from air-cooled general-purpose datacenters to liquid-cooled AI factories. Present the thesis that the frontier bottleneck is dynamic control of the full cooling path, not heat removal alone.

2. AI Rack Density and the Move to Direct-to-Chip Cooling

Use Google, NVIDIA, Schneider Electric, ASHRAE, and OCP to show that AI racks are becoming too dense for conventional air cooling. Explain why warm-water direct-to-chip cooling is the practical near-term architecture.

3. Why Steady-State Chip Temperature Is Not Enough

Explain the transient nature of AI workloads and the rack-to-facility delay chain. Define the difference between controlling chip temperature and optimizing the whole thermal-fluid system.

4. Evidence for Workload-Aware Cooling

Review coordinated cooling and compute management, predictive liquid cooling, dynamic flow control, and dynamic thermal management literature. Distinguish peer-reviewed evidence from vendor-reported claims.

5. Hypothesis and Benchmark Design

State the hypothesis and define the proposed benchmark. Describe workload cases, system boundaries, control variables, and evaluation metrics.

6. Methodology

Compare fixed control against workload-aware co-control. Describe simulation, validation, sensitivity analysis, and failure/degradation scenarios.

7. Expected Results

Predict where co-control should help most and where it may not. Emphasize conditions under which the hypothesis could be falsified.

8. Counterarguments and Limitations

Discuss operational complexity, proprietary traces, site variability, safety, and standardization.

9. Conclusion

Restate that the frontier is not liquid cooling in general, but validated transient control of liquid-cooled AI infrastructure.

Disclosure: ChatGPT (OpenAI) was used to support brainstorming, organization, and initial evidence gathering.

Initial Evidence Base

• ASHRAE/NEMA/PNNL AI Data Center Energy Performance Framework: frames AI datacenters as energy, water, grid, design, retrofit, commissioning, and operations problems.
• Open Compute Project AI Infrastructure Standards: identifies power, cooling, telemetry, and control interfaces as urgent standardization needs for AI racks.
• Google Cloud on 1 MW racks and liquid cooling: shows hyperscale movement toward megawatt-class racks, CDUs, and liquid cooling.
• NVIDIA Rubin liquid cooling discussion: supports the warm-water and dry-cooler direction for AI factories.
• Schneider Electric direct-to-chip cooling blog: explains D2C architecture and emphasizes integrated power, cooling, controls, and operations.
• Coordinated Cooling and Compute Management for AI Datacenters: directly supports joint compute and cooling control for AI workloads.
• ProphetStor predictive liquid cooling: industry example of workload-aware liquid-cooling telemetry and pump/valve control.
• Dynamic direct-liquid cooling reliability work: supports the feasibility of dynamic flow control in direct-liquid cooling.
• UTA dissertation on hybrid-cooled servers: supports the move from component studies toward coordinated, energy-aware system architecture.
• Toward Physics-Informed Machine Learning for Data Center Operations: supports multi-scale modeling that connects chip-level and facility-level thermal behavior.