Exoscale Trials Innovative Cooling Solutions from Diggers for Data Centers

Exoscale, a notable player in the cloud hosting sector, is currently evaluating an advanced cooling system developed by the Austrian specialist Diggers. This innovative system aims to optimize energy efficiency in AI computing by integrating direct and indirect water cooling methods within closed environments.

Designed to operate without the need for external air cooling, Diggers' system promises to utilize up to 98% of server waste heat while simultaneously reducing cooling efforts by 50%. The configuration accommodates four chassis-free servers, each equipped with dual CPUs and three GPU modules, all contained within a compact structure akin to half a standard server rack. The design allows for vertical stacking of two units, with each box functioning as a self-contained system featuring its own water circulation, connected to a secondary cooling circuit. This setup ensures enhanced protection, achieving an IP67 rating, and allows for operational temperatures of up to 50 °C, with successful tests conducted at even higher temperatures.

The cooling mechanism employs Direct Liquid Cooling (DLC) heat sinks to extract heat from primary heat sources. Additional waste heat from other server components is managed through eight air-to-water heat exchangers, strategically positioned behind the servers. These exchangers are equipped with fans to facilitate airflow, directing it from the front to the rear of the server cabinet, ensuring efficient heat dissipation.

Heat transfer occurs through a secondary water circuit, eliminating the need for traditional cooling methods such as raised floors. This makes Diggers' cooling systems versatile enough for use in various environments, including rugged outdoor locations when configured to the IP67 standard.

To further enhance safety, the system incorporates a monitored vacuum pump that maintains pressure within the water lines. In the event of a leak, the pump draws air into the lines, effectively preventing water loss. In instances where the pump operates beyond its typical usage of approximately 20 minutes per day, an alarm is triggered.

Additionally, Diggers has implemented a unique approach to GPU cooling. The three GPU modules are positioned approximately 10 cm above the mainboard and are fitted with large aluminum heat sinks. The use of aluminum, known for its corrosion resistance and lower recycling energy costs, presents a sustainable alternative to copper. Currently, the GPU heat sinks connect to the water circuit via flexible hoses, with plans to integrate them more permanently in future designs.

Embracing principles from the Open Compute Project, Diggers has minimized resource usage by eliminating traditional server enclosures. The initiative promotes energy efficiency through fewer large-diameter fans, a method that has proven effective in OCP hardware. Diggers is also exploring models with integrated power supplies that deliver 12 volts directly through power rails, adhering to OCP guidelines.

Exoscale is currently testing these systems in their Floridsdorf data center in Vienna. This facility operates without a raised floor or traditional cooling systems, relying solely on connections for electricity, network, and water. The company has confirmed the energy savings claimed by Diggers.

Feedback from the GenLearning Center at the École Polytechnique Fédérale de Lausanne, which utilized the systems during a recent AI competition, highlighted the exceptional performance and stability of the setup. Participants were able to seamlessly access advanced large language models without experiencing any downtime, underscoring the effectiveness of Diggers' cooling technology.