- 1MW racks are coming soon and represent an exponential jump in rack power levels
- These new racks will require robust liquid cooling systems
- They'll also need new physical designs to separate power distribution and compute
When Flex President Chris Butler started talking about the imminent reality of 1 megawatt (MW) racks in an interview this week, it sounded like an echo. That’s because just two days before LiquidStack’s Head of Strategy Angela Taylor mentioned the same thing.
According to Butler, they’re coming soon.
“I think in 12 to 18 months, it’s going to be a reality at scale,” Butler said. While scale is relative and these won’t replace every rack in the data center, he noted there are “large scale plans to roll this out for all the GPU applications.”
For context, there are 1,000 kilowatt (kW) in a MW. That means 1MW is a wild leap from the 15 kW less racks that permeate data centers today. It’s even a giant jump from the high-performance 40-100 kW rack power levels people initially thought AI would require.
But Nvidia roadmap for 2027 includes GPUs that will require 600kW of power per rack. So, cloud providers are thinking big to prepare for the future. Very big.
Keeping cool
The thing is, that kind of leap has huge implications for power and cooling systems.
As Butler put it, when it comes to cooling 1MW racks, “You just can’t do it with air.” Why? Because these kinds of systems put out enough heat to warm multiple houses. That then means bringing in liquid cooling systems and designing data center infrastructure accordingly.
Given Flex now owns JetCool, Butler obviously has an interest in insisting liquid cooling is required. But Dell’Oro Group’s Research Director for Data Center Physical Infrastructure Alex Cordovil agreed.
“Liquid cooling will be absolutely mandatory to support IT loads of such densities,” he told Fierce.
Thus, vendors across the spectrum are preparing for the inevitable. Enter Taylor’s comments about a new modular coolant distribution unit (CDU) that LiquidStack has designed to allow cooling capacity to changing rack needs. The new CDU comes in 2.5MW modules in a system that is expandable up to 10MW.
“We’re now talking not just about 600 kilowatts but 1 megawatt racks. So you’ve got that to contend with and how do you design your cooling systems such that you are making an investment today for what you need and also where you see it going. So, that’s really why we chose to do it in a modular fashion,” she said.
Cordovil said single-phase direct-to-chip systems – which are currently the most popular and the variant that LiquidStack’s new CDU supports – are expected to continue to evolve to meet rising power demands. For now, the expectation seems to be that 1MW racks will be within their scope.
But he noted it’s not yet clear “at what rack density one-phase systems will hit their ceiling.” And when they do, that’ll open the door for dual-phase direct liquid cooling systems, like those offered by Zutacore and Accelsius.
Power up
Cooling systems aren’t the only thing that needs to change to enable 1 MW racks. Power supply systems are another critical component.
Flex is currently working on 400 volt (V) direct current (DC) systems, and Butler said it’s already eyeing 800V DC and even 1500V DC for the future.
As we’ve noted before, the shift from today’s 48V systems to 400V DC requires a fair bit of safety enhancements and worker retraining. But Cordovil and Butler said it’s also set to change physical rack designs.
Today, it’s common to have power distribution shelves and compute servers in the same rack. However, the move to exponentially higher power levels means power and compute may soon have separate homes.
“A collaboration between the likes of Google and Microsoft is working on a new design with the Open Compute Project Foundation, codenamed ‘Mt Diablo.’ This is pushing the entire power architecture to an adjacent rack sidecar allowing for more servers to be packed within a same rack,” Cordovil explained.
He added that freeing up space in the server rack is “essential to allow for more north-south fast communication within the rack and pile up more IT load.” It is this shift that will allow the industry to get to the “mythical 1 MW” rack, he concluded.
Of course, rising power levels inside the data center have external implications for utilities and the like. Read more about power generation and sourcing side of the equation here.