Skip to main content

The AI Hardware Supplement May 2025

Page 1


The AI Hardware Supplement

Why wait to build an AI data center when you can unpack one now!

Are you ready to revolutionize your AI infrastructure with modular data centers?

Modular AI data center from Schneider Electric present a prefabricated, scalable infrastructure solution tailored to meet the extreme power, cooling, and density demands of generative AI workloads. Discover how prefabricated modular data centers can accelerate AI adoption, simplify design and construction, and support sustainability strategies.

Download the e-guide and reference designs to learn more how Schneider Electric’s Modular AI Data Centers address these critical challenges and help you stay ahead in the evolving technological landscape!

Power Module
High performance power system that integrates switchboard, UPS, with Li-Ion battery cabinets, PDUs, and optional backup generation set.
Genset
Chiller

Contents

4. Building infrastructure for a systems-level world

Arm’s SVP and GM of infrastructure explains how the company has gone from playing catch-up to being a market-leading triple threat

8. Advertorial: Up, up, and AI-way Driving next-generation data center infrastructure while AI takes flight

10. Laying the cooling roadmap for the future of semiconductors

How Coolerchips hopes to allow bigger, denser chips

13. AI is making the network ‘sexy’ again

Networking is an important piece of the AI data center puzzle

Hardware’s paradigm shift

The AI revolution that is sweeping through the data center industry means operators need to reconsider the hardware that brings their data halls to life.

From chips to cooling systems to the networking cables that ensure data flows smoothly between servers, planning a data center in 2025 is a different proposition when compared with even two or three years ago.

At the heart of these changes is a quest for efficiency, to enable servers to become more powerful and deal with the complex workloads that AI throws up, as well as making best use of power resources which are in short supply in many markets.

For Arm, the British chip company that developed the semiconductor designs which power mobile devices such as the iPhone, the data center seemed out of reach for many years, with Intel’s rival x86 architecture ruling the roost.

Multiple attempts to get Arm into the data center have fallen flat over the years since the company spun off from Acorn computers, but AI appears to have shifted things in its favour, with the hyperscalers turning to the company’s low power architecture for their own in-house chips, putting Intel on the back foot in the process.

For this supplement, Charlotte Trueman spoke to Mohamed Awad, SVP and GM of Arm’s infrastructure business, about the company’s Neoverse range, which

he believes can give users both power and efficiency advantages.

AI has helped make liquid cooling, for so long tomorrow’s technology in the data center, a reality for many operators, with leading edge GPUs requiring water-based systems to ensure they don’t overheat.

As thermal design points rocket, new cooling technologies will be required, and the US DOE is attempting to get ahead of the game with its Coolerchips initiative. The $40m ARPA-E R&D program aims to bring these new systems to life, and projects being supported include concepts from Nvidia, Flexnode, HP, and Raytheon.

Dr. Peter De Bock, ARPA-E’s outgoing program director, gave Sebastian Moss an update on Coolerchips, and where the projects could go next.

Making networking sexy again may seem like a task beyond even the most advanced AI system, but, as Ciena’s Jürgen Hatheier tells Dan Meyer for his feature on AI networking, the way servers are connected has become a hot topic in the data center industry once more.

With AI workloads set to make up an increasingly large portion of data center activity in the coming years, sales of networking equipment could rocket, too. In his feature, Dan explores the changing face of networking in the AI era, with new ways of connecting servers, both within data centers and between different sites, on the horizon.

Building infrastructure for a systems-level world

Arm’s SVP and GM of infrastructure Mohamad Awad explains how the company has gone from playing catch-up to being a market-leading triple threat in 15 years

Charlotte Trueman Compute, Storage, and Networking Editor

Over the past 15 years, chip design giant Arm has been on quite a journey within the infrastructure space, Mohamed Awad, SVP and GM of Arm’s infrastructure business, tells DCD.

What started as a joint venture between Acorn Computers, Apple, and NXP Semiconductors, Arm – an acronym of Advanced RISC Machines – was officially born in 1990, with the company starting to license its processor IP in 1993.

A $32 billion takeover by SoftBank followed in 2016, with Cambridge-based Arm being delisted from the London Stock Exchange. It went public again in 2023, though SoftBank remains its biggest shareholder.

When it comes to chip architecture, Arm now dominates the market, but that wasn’t always the case. Traditionally, Intel had the server market cornered with its x86 chips but, as hyperscalers increasingly look to develop their own CPUs, Arm’s offering has grown in popularity as it allows companies to design a chip that can be customized to meet their exact needs.

Arm’s initial focus was on offering general-purpose computing solutions and ensuring its software allowed for frictionless deployment, but in 2018, the company shifted focus, launching its Neoverse-based CPUs.

Designed specifically for data center, Edge, and HPC use cases, Neoverse allowed Arm to pivot from simply providing the basic building blocks for infrastructure solutions – or offering cores which were designed for some other purpose, like mobile, but could then be put together into a system-on-chip (SOC) – to developing technology specific for infrastructure.

Arm divides its Neoverse offering into three separate groups, but Awad says the product line can primarily be broken down into two main product lines: The V series, which focuses on high-performance, general-purpose compute, and the N series, which focuses on the server market. The company also offers an E series for Edge compute.

Arm’s V series – V2 specifically – is what AWS Graviton Four, Google

Axion, and Nvidia Grace are based on, while Microsoft Cobalt’s chip is a product of the N series.

While Awad jokes that AI is taking “a lot of oxygen out of the room” at present, he says HPC is still “grinding away,” and Arm is continuing to see quite a bit of traction with such workloads.

“The V series is very popular in the HPC space, and Graviton, Grace, and Axion are all based on the V2 series, while I think the N series is also popular, but more with cloud-type workloads or networking use cases,” Awad says. “Our N series has more of a focus on efficiency. It’s great for workloads where you’ve got lots of parallel workloads happening simultaneously.”

“More of the ecosystem… is moving to a software-defined infrastructure that has a level of commonality across that entire compute fabric”

When it comes to the N series, while the focus is often on the server market – “when you talk about data centers, it’s certainly a very important part of the story” – Awad notes that across the industry there have been a number of transformations with other component parts in recent years that have seen the conversation move away from being solely focused on compute, particularly as products such as DPUs have entered the market and forced changes in areas like networking as a result of the connectivity challenges associated with

infrastructure perspective, we’ve made sure that our N Series and V Series of products exist across that entire spectrum,” Awad says.

“More and more of the ecosystem or the infrastructure is moving - and this has been a transition that’s been going on for a decade - to a kind of softwaredefined infrastructure that has a level of commonality across that entire compute fabric.”

Consequently, when it comes to architecture, Awad says that understanding these changes is incredibly important because organizations increasingly want to add acceleration across the fabric, be it networking, storage, or servers, to allow themselves to keep up.

Changing the game

When Neoverse was first released, Awad says it was about allowing companies to leverage Arm’s heritage of performance and efficiency to provide immediate TCO gains within the context of a software ecosystem that “wasn’t perfect back five or six years ago, but was good enough for them to get going.”

However, in the last couple of years, Arm decided it wanted to go further than just providing an architectural license that would still require a lot of work and investment to build something on, and has evolved Neoverse in order to meet the changing needs of its customers.

“Firstly, we decided to say, ‘okay, we’re going to hand you the CPU and the interconnect, and you can stitch it together,’” Awad says. “That’s a lot less work than just handing over an architectural spec, but we were aware it still required a meaningful amount of work.

“Networking is evolving, storage is changing in a meaningful way, even wireless base stations are changing, so, from an

“Then we took that approach to compute subsystems, where we actually take the CPUs and the interconnect and stitch them all together ourselves. This means we can hand over an integrated, verified subsystem that can then very quickly be customized for specific use cases, allowing you to optimize the silicon around your data center without enormous amounts of investment.

“That has been a game changer, and you can see it with the likes of Microsoft Cobalt, where, very quickly, the team

has been able to show up with a solution and market by leveraging our compute subsystems.”

As a result, from Awad’s perspective, the last five years have been “nothing short of amazing.”

He says: “It started with AWS, with Graviton, but today, every major hyperscaler, whether that’s Google, or Microsoft, or OCI, they are all building or deploying silicon based on Arm in some way.”

Awad says that currently, AWS has tens of thousands of customers running on Graviton, noting that more than 50 percent of the compute deployed by the company in the last two years has been Arm-based. He says that while there are arguably a number of different reasons why that might be, it is Arm’s power efficiency and low barrier to entry – “simplifying what it takes to go build

“The dream is to get to the point where workloads easily migrate from one end of that infrastructure to the other, based on… where the compute capacity exists”

silicon” – that has really set it apart.

“The world, moving forward, it really is a systems level world, and whether you’re talking about one of those hyperscalers, or you’re talking about a technology provider like Nvidia, [everything they’re building] is based on Arm because [with AI workloads]

the primary workload starts on the CPU and is then offloaded on to the accelerator, whether it’s homegrown or not.

“It’s actually a very interesting position that we’re in. We’ve gone from 15 years ago, trying to catch up with the modern workloads, to now, in many cases, seeing those key AI workloads actually being optimized for Arm first… which is a great place to be in.”

The next big thing in silicon

When it comes to addressing the main challenges facing today’s data centers – in particular AI data centers – Awad says the line between power and performance has become “so blurry” given the scale that some of the AI systems are operating at.

“The two go hand in hand… if you have unlimited power, you can have unlimited performance, it’s just about how much silicon you want to throw at it.

“The reality is… power equals performance at the end of the day. However, right now, the thirst for performance is so great, it’s not like saving power actually means consuming less power, it just means adding more compute. That’s the world that we’re living in today.

“Now, that’s going to change at some point, but today, it’s really just about getting as much compute as you can, so wherever you can save power, it becomes incredibly important. And frankly, tearing the barriers down so that you can cram more performance into a given power envelope, that’s really what it’s all about.”

And from Arm’s perspective, these are all things which you just can’t do with offthe-shelf silicon.

“Infrastructure itself is evolving in a massive way right now, whether that’s AI or otherwise, and more and more of these

big hyperscalers are looking at their data center infrastructure which, in the main, has been general-purpose commodity, off the shelf legacy architectures or CPUs, and realized that while that used to be good enough, it isn’t any more.

“Instead, it’s all about making sure that everything is optimized, as that’s the only way that they’re going to get the sort of efficiency that they need or scale to the level of performance they want.”

In 2025, the majority of investment in infrastructure is tied up with a very small number of technology providers, most of whom are working with Arm in what Awad describes as a “meaningful way.” He argues that this is due to the fact that when it comes to providing evergrowing levels of compute in today’s power-constrained environment, having customizable silicon that is designed around your data center or software stack, as opposed to a decade ago where you would “take whatever I had off the shelf and just try to cobble something together, which technically works, but is certainly not going to be as efficient.”

Awad says Nvidia’s Grace Hopper system is one of the best examples of this. Where, historically, a number of GPUs would be placed on a PCIe bus and connected to a general purpose x86 processor “which works, but you’re limited in terms of the memory you have access to, and the connectivity between those devices based on that PCIe interface,” Grace Hopper, and by extension Grace Blackwell, creates a coherent memory domain between the CPU and the accelerators, providing a much higher speed link between those two devices.

“We’ve gone from 15 years ago, trying to catch up with the modern workloads, to now, in many cases, seeing those key AI workloads actually being optimized for Arm first”

For the myriad of companies that have emerged in recent years, professing a desire to take on the incumbents and do “the next amazing thing with silicon,” Awad says that there are three things which set Arm apart: the performance and efficiency of its designs, being part of a broad ecosystem that people can easily leverage, and providing customers the freedom to customize and augment the hardware with capabilities like acceleration, interfaces, or connectivity.

He adds that the company’s success has endured because of its reputation for working with so many hyperscalers, making new customers feel secure that what Arm has to offer will be here for the long term, as so many big names have staked everything on building hardware based on its IP.

However, while Awad acknowledges that innovation takes time, he says that the industry is already starting to see it emerge, largely driven by the promise of having infrastructure that is optimized from the point where data is onboarded all the way through to the data center.

“The dream is to get to the point where workloads easily migrate from one end of that infrastructure to the other, based on where it can be processed most efficiently or where the compute capacity exists," he says.

“We’re marching towards that. I’m not saying it’s going to happen in 2025 but that is certainly where the world is going to head long term, and it’s the only way you can really bring out the necessary levels of efficiency and performance from every ounce of compute, because at the end of the day, technology that’s sitting idle isn’t doing anybody any favors.”

Up, up, and AI-way

Driving next-generation data center infrastructure while AI takes flight

As artificial intelligence (AI) takes off at an ever-accelerating pace, it brings with it an urgent need for infrastructure that can keep up.

GPU chips, essential for training and running AI models, are pushing the limits of data center power densities. Though AI is still in its early ascent, Nvidia’s rapid development of new chips demonstrates soaring demand for high-performance computing. However, meeting this demand comes with complexity.

Nvidia’s leadership in developing these powerful GPUs highlights the role of hardware in driving AI progress, which in turn necessitates robust infrastructure solutions that can handle unprecedented power and cooling requirements, like those that Schneider Electric provides.

To shed light on how Schneider Electric is helping data center operators address these challenges, DCD spoke to senior systems design engineer Allegia Wiryawan to get her take on enabling the AI revolution.

Building the aircraft while flying it

As higher rack densities become the norm, data center owners are upgrading their equipment to match this increased capacity. However, scaling electrical gear requires careful attention to many different factors, for example, changes in short circuit ratings: “The short circuit current reaching these higher density IT rows tends to be higher, but some of your

IT equipment may not increase its short circuit rating proportionally, so you might need to adjust your electrical protection design to accommodate that,” says Wiryawan.

On the cooling front, rising densities are driving a shift toward liquid cooling, which is now being deployed at scale:

“Since liquid cooling is still relatively new, there aren’t many established standards or guidelines yet – we’re actively developing and refining them as we go.”

Navigating new challenges with familiar tools

With the rapid pace of chip development, one might wonder whether there’s a risk of misalignment between what’s being built and the infrastructure needed to support it. According to Wiryawan, that gap isn’t as wide as it might seem: “While the designs are evolving, we’re largely adapting existing, well-understood components.”

While deep expertise in data center design resides with a relatively small group of specialists, it’s increasingly accessible through collaboration with hyperscalers, consultants, and industry forums, helping the ecosystem keep pace with chip innovation.

This approach allows Schneider’s teams to build on proven technologies, minimizing knowledge gaps. Although new challenges – particularly around cooling – do arise, they are manageable and not entirely unfamiliar.

Straight from the source

Schneider’s close collaboration with Nvidia offers unique insights into Nvidia’s roadmap and direct access to the company’s requirements for designing GPU racks. This partnership allows Schneider to tailor its infrastructure solutions to effectively support Nvidia’s cutting-edge GPUs.

“There’s a continuous back-and-forth validation process where we design to their specifications and provide feedback on the challenges we encounter and how we overcome them,” explains Wiryawan.

Schneider and Nvidia aim to bridge the market’s limited expertise by providing publicly available, validated reference designs – crucial for helping data centers navigate rapid growth and complexity.

On the cooling side, Schneider leverages its advanced modeling software, including EcoStruxure IT Design Computational Fluid Dynamics (CFD), which simulates airflow in both air and liquid cooling environments. Complementing this,

Schneider’s Ecodial and ETAP platforms validate electrical designs through short circuit and load flow analysis.

In simple terms, these platforms help data center designers and operators predict how air will flow and heat will distribute, enabling them to optimize cooling efficiency, equipment placement, and energy use well before construction begins.

Why Schneider’s designs lead the pack

According to Wiryawan, Schneider’s reference designs stand out for their exceptional depth and completeness. Unlike other vendors, Schneider delivers comprehensive site-wide electrical and mechanical documentation – including oneline diagrams, piping layouts, and validated ETAP and EcoStruxure IT Design CFD models.

“We also rely on advanced in-house engineering platforms like Ecodial and EcoStruxure IT Design CFD. Plus, we’re leading the way in developing control system reference architectures, a fact Nvidia has recognized. Our designs even simulate high-density AI clusters in real-world conditions, demonstrating impressive PUEs of 1.14 in San Francisco and 1.17 in Dallas.”

Beyond reference designs, Schneider offers its ‘grid-to-chip ETAP digital twin’ – a collaborative innovation with ETAP and Nvidia designed to help customers understand and implement changes in power infrastructure.

This digital twin models and simulates the entire electrical path from the utility grid down to the chip, tailored specifically for high-density AI workloads Given how these workloads differ from traditional ones, the platform equips designers and operators with critical insights to accurately size equipment and manage key power parameters.

Ultimately, it enables smarter decisionmaking and smoother facility modernization.

A world-first digital twin

With AI workloads carrying very different characteristics from their traditional counterparts, existing modelling platforms have been ill-equipped to handle the heat, both literally and figuratively. The ETAP electrical digital twin solution, as Wiryawan describes, is a world-first for such environments, largely thanks to the

groundbreaking innovation and integration between Schneider, ETAP, and Nvidia: “From Schneider’s side, we’ve been working closely with ETAP to ensure their software includes all the latest parameters and models of our electrical equipment, allowing for more accurate analysis. Nvidia brings their Omniverse platform, adding powerful 3D simulation and visualization capabilities, enhancing the digital twin aspect.”

This collaboration brings together best-in-class capabilities: Schneider’s deep domain expertise in power infrastructure, ETAP’s advanced electrical system modeling, and Nvidia’s real-time simulation and AI workload insights. Together, they create a more robust and predictive design environment than anything else currently available.

"This kind of insight helps ensure the design supports both performance and energy efficiency"
>>Allegia Wiryawan, Schneider Electric

The three core elements of AIenabled data centers

When designing AI-enabled data centers, Schneider Electric highlights three key pillars that must work together to support the high-performance, high-density demands of AI workloads:

• Power: AI systems consume significantly more electricity and have different power profiles than traditional workloads, making reliable, scalable power infrastructure critical to handle dense GPU clusters, ensure uptime, and avoid bottlenecks.

• Cooling: AI chips generate a lot of heat, which traditional air cooling systems may not efficiently manage. Advanced cooling methods like liquid cooling are often needed to maintain performance and prevent overheating.

• IT room/space: The physical layout of the data center must support dense compute environments. This includes optimizing rack placement, airflow, cable management, and equipment accessibility to ensure efficient operation and maintenance.

Providing a real-world example, Wiryawan illustrates the kind of visibility EcoStruxure IT Design CFD can provide: “One of our engineers used the platform and found that standard hot aisle widths weren't sufficient for AI workloads. Based on airflow modeling, we determined that wider hot aisles were needed to accommodate the higher thermal output. This kind of insight helps ensure the design supports both performance and energy efficiency.”

Full lifecycle control systems

During the design phase, Schneider uses platforms like ETAP, Ecodial, and EcoStruxure IT Design CFD to virtually validate power and cooling systems in its reference designs. This ensures the designs are optimized to support AI workloads efficiently and meet key performance, efficiency, and safety targets – well before they are deployed in realworld data centers.

Once the facility is running, building management (BMS) and energy and power monitoring (EPMS) systems like EcoStruxure and AVEVA are used to collect and monitor real-time data from power and cooling systems. This helps operators confirm the facility is performing as it was designed to, as well as the ability to detect failures early, predict maintenance needs, analyze, and optimize ongoing performance.

Building AI-ready infrastructure

While AI delivers major benefits, it also introduces new infrastructure challenges. Schneider’s reference designs offer a reliable starting point that can be adapted to specific needs, enabling more rapid and streamlined AI deployments.

Combined with design software like ETAP and EcoStruxure IT Design CFD, these platforms allow you to validate your setup before making costly investments, reducing the risk of design flaws later on.

View more information on Schneider Electric’s available AI reference designs here

Laying the cooling roadmap for the future of semiconductors

How Coolerchips hopes to allow bigger, denser chips

When the US Department of Energy (DOE) launched a program to radically improve the cooling capabilities of the data center industry in September 2022, it appeared to be a long-term effort to get ahead of where semiconductors could go over the next decade.

Two months later, ChatGPT launched, chip thermal design points (TDP) began to spiral, and the program became about simply keeping up.

“In the first meeting when we launched the Coolerchips project, people said ‘126 kilowatts per rack is crazy,’” outgoing ARPA-E (Advanced Research Projects Agency–Energy) program director Dr. Peter De Bock recalls. “Nobody has ever said that to me again.”

In May 2023, the DOE announced that 15 distinct projects would be funded

through the $40 million Coolerchips program, including concepts from Nvidia, Flexnode, HP, and Raytheon.

“The way the portfolio was selected, you saw a lot of diversity,” De Bock says. “That means we selected diverse paths, although most of them were doubled: So we have two on two-phase, two on single-phase, there’s some redundancy, but in the end, they can’t all meet the goals.

“My goal is to make program targets work. It’s not to make every team successful.”

The time to shine

After a year of prototyping, PowerPoints, and lab experiments, the projects now face a critical juncture: They have to produce a working server featuring their cooling solution this summer. “Some teams will not meet it, and their funding

will be stopped,” De Bock says. “We will be thanking them kindly.”

The remaining projects will then be able to make the case to win any of that funding. “You can see the sheer sweat and focus of the teams to get to this point, but it’s where we need to be - you can’t just talk, you have to walk the walk.”

Once they reach that milestone, they then have a year to develop a partial rack. “They don’t have to build me a full rack because it’s too much equipment, but at least probably three or four slots filled, 10 kilowatts of compute per rack, that way they have to solve the system integration challenge,” De Bock says.

With both goals, entrants have the option to use the data center site at the National Renewable Energy Laboratory (NREL) Flatirons Campus in Colorado, home to one of the world’s most energy-

efficient data centers.

About half the projects have applied to make use of the site, where flow rates and inlet temperatures can be carefully controlled. Others have their own labs, or are deploying with customers.

De Bock can’t pick favorites, but DCD still asks about which technological approaches appear to be leading. “I think that’s one of the bigger understandings,” he counters.

“I have a lot of people ask me, ‘Is immersion the future? Is two-phase the future? Is it direct to chip?’ And I’m telling them, all of these projects have the potential to meet it, but the actual winner will be the one that makes the most economical, most maintainable, most reliable box. It’s not the technology under the hood. It’s the box. It’s like VHS and BetaMax, right? It doesn’t have to be the best technology. It has to be the best product. That’s what we’re looking at.”

Commercialization of the solutions is a core part of how each effort is gradedentrants need to show that, not only can their tech cool as well as they say, but that there is a roadmap to manufacture

Avoiding computer chips turning into potato chips

Power density and TDP conversations often miss what is actually happening at the chip level, De Bock says. “One of the things most people don’t talk about is that chips are getting bigger.”

Nvidia’s latest and greatest chips are actually two reticle-sized GPUs sandwiched together. The Rubin Ultra, slated for release in late 2027, has four reticle-sized GPUs, dramatically increasing the size of the single chip. Rival Cerebras offers wafer-scale-sized mega chips, while others are believed to be considering moving to similar scales.

“These chips are going to grow, and our ability to bond to the chip with a cooler becomes harder and harder,” De Bock explains, as the chip expands at a different rate than the metal coldplate above it. “Chips, when they get very large, they like to potato chip and warp a little bit.”

"I don’t care about your exotic solution. Exotic was fun. Now you’ve got to make it work”
>> Dr. Peter De Bock

however. Just as Nvidia and others have argued that a GPU should no longer be considered a single system, but rather just a node in a facility-scale computer, De Bock calls for a rethink of cooling at the facility level.

“Some teams will not meet it, and their funding will be stopped. We will be thanking them kindly”
>> Dr. Peter De Bock

and install it at the scale expected of this exploding industry.

“I don’t care about your exotic solution,” De Bock says. “Exotic was fun. Now you’ve got to make it work. The twophase teams are having a harder time, but they also realize that if they’re successful, their ability to go to even higher power loads is greater for them.

“You can see that JetCool, which is already a proven startup - and has done really well, getting acquired by Flex - but they have a certain amount of capability, and we’re helping JetCool with a 2.0 design to stretch them into even higher power densities.”

In hopes of avoiding such potatification, the program includes “several coldplates and coolers that are not made out of copper or aluminum,” De Bock says. “They’re made out of silicon or copper-tungsten, which has a similar coefficient of expansion, so they can bond to the chip in a much better way.

“The coefficient of thermal expansion is three for silicon; for aluminum, it’s 25. That means that as your chips get hotter, these things slide because they grow more, and with these high heat fluxes. The cold plates that are coming out of Coolerchips are naturally matched with the chip in its coefficient of expansion.”

He continues: “All of these technologies will push us past the 100kW and on paths to 600kW or more. This is all good stuff.”

Staying at the chip’s surface is a flaw,

“I do see a challenge - cooling companies want to work from the building roof down to the CDU,” he says. “And then the chip makers want to go work with OEMs and the server. And there’s a gap in the middle, the no-man’s land; who wants to own the connection between the coldplate and the CDU? There has to be some co-design, and that’s the part we have to work on.

“I do think that maybe we need to explore how that can be solved as a system. Would the CDUs and the coldplates become a system - does that get owned by the OEM? Or does it get owned by the cooling maker? That is something where, as an industry group, we have to resolve.”

As it stands, he believes that cooling companies generally avoid touching the server because “then they’re liable for the warranty on the server and the service level, the SLAs, and all that.”

Fixing this is “key to unlock all the technology,” he says, adding that he is working with insurers to help them understand how to cover cooling systems. “My background is aerospace, and I have an aerospace-level focus for reaching extreme reliability of the cooling

system, and we have to develop the right mechanism to credit them for it.”

As the program enters its latest phase, De Bock is bullish about the next steps. “I think there are now probably 200 people focused on this program to move this whole field forward. This is a wave of momentum to get us ready,” he says.

The

elephant in the room

He’s less forthcoming about the current administration’s approach to staff retention, declining to comment

on whether Coolerchips has lost any personnel thanks to rounds of layoffs, voluntary redundancies, and chaos as part of the DOGE-led dismantling of the administrative state.

“The US government has announced great support for Stargate Project,” De Bock says, pointing to a $500bn privatelyfunded data center project for OpenAI that President Donald Trump announced on his first day in office.

“And one thing you can see is that the Secretary of Energy, Chris Wright, came to the ARPA-E summit to see some of the work going on with Coolerchips.”

What Wright saw was De Bock’s pitch for a system-level approach to chip cooling. “I call this the power-cooling nexus. With the density of the rack [set to] exceed entire neighborhoods, not having a cooling system that’s an equal level of sophistication can be a real risk.

“You’re pushing a lot of energy into something, so you need to be able to remove it at the same speed and same efficiency. Bringing the cooling systems

“The two-phase teams are having a harder time, but they also realize that if they’re successful, their ability to go to even higher power loads is greater for them”
>> Dr. Peter De Bock

to their next level is what this program is all about.”

De Bock, meanwhile, is himself leaving the program. Similar to DARPA (Defense Advanced Research Projects Agency), the agency intentionally term-limits directors to ensure rapid progress and new ideas.

“It’s been one of the most exhilarating and meaningful chapters of my career,” he says. 

AI is making the network ‘sexy’ again

Networking is an important piece of the AI data center puzzle

Artificial intelligence (AI) is causing a real impact on networks, forcing data center and network owners to re-evaluate their current technology choices and how they will be positioned to face an expected onslaught of AI-generated networking demands. The actual mass of that onslaught remains nebulous, but most agree it will be substantial and will require significant financial investments to support.

Most of this initial AI-related traffic generation is happening within data center fabrics. These are the intra-connected links that tie together the thousands of compute boxes running AI workloads within a single data center location. This traffic demand is what is driving significant investments into building out new and updating legacy data center locations.

“Certainly, AI is sort of the overarching theme behind a lot of the things that are driving capacity in customer networks, whether it's hyperscalers or service provider, and ultimately enterprise networks as well,” Bill Gartner, SVP and GM for optical systems at Cisco’s Optics Group, said during a 'fireside chat' session at the

recent OFC 2025 Conference.

Dell’Oro Group in a recent report found that global data center capex increased 51 percent in 2024, hitting $455 billion in investment last year. The research firm noted that custom accelerators from hyperscalers like Amazon Web Services (AWS), Microsoft, and Google Cloud accounted for more than half of that total investment, and most of last year’s growth was fed by those hyperscalers bolstering their support for AI training workloads.

Matt Landek, who leads real estate consulting firm JLL’s data center project development and services, said that his firm is currently quoting only around 15 percent of total data center workloads are AI-related, but that it expects that number to hit around 40 percent by 2030. Analyst firm LightCounting last year predicted that growing AI use will result in a doubling of sales for Ethernet optical transceivers used in AI clusters.

AI driving inter-data center networking

There is also growing demand to bolster inter-data center networking connections

to help support AI-related data inferencing outside of those main data center locations. This is typically targeted at improving the performance of optical network connections between large data centers in different regions or to Edge locations that are closer to end users, where most think data inferencing will take place.

These investments are typically more involved than intra-data center updates as they include more complicated equipment in less controlled environments, but there are considerable efforts in place to boost these connections. Large players like Huawei, Ciena, and Nokia hold sway in terms of market share, with each also pushing out higher-speed options using advanced fiber optic and transceiver

“I think there's a whole new breed of optics that are going to be needed”
>> Rob Shore, Nokia

technology that can support speeds up to 1.6 Tbps.

Dell’Oro Group noted that the optical transport market ended last year “on a strong note, growing approximately 45 percent quarter-over-quarter” during the final three months of the year.

“There is light at the end of this tunnel,” Dell’Oro Group VP Jimmy Yu wrote. “The optical transport equipment market entered a period of revenue contraction in late 2023 and looks to be exiting it in early 2025. The strong fourth quarter growth rate in 2024 is a great indication that supply and demand for optical transport equipment is near equilibrium, meaning the equipment glut is over.”

Those market leaders are jumping at this new opportunity.

“I think we are in an unprecedented

“Networking has become one of the sexy things again”
>> Jürgen Hatheier, Ciena

industry disruption, and what we have seen over the last couple of years is just a commitment to build compute, and while networking had a little bit of a back seat, the conferences over the last 12 months have completely changed that,” says Jürgen Hatheier, CTO for Ciena’s International business. “Networking has become one of the sexy things again, where hyperscalers have to pay attention not only to the fastest Nvidia GPUs and the latest Google TPUs. They also need to build an ecosystem. And while it has been built a little bit by brute force within the data center, with more and more capacity, there is now a lot more sophistication in how it is being built.”

Hatheier’s “sexy” comment was echoed by Nokia’s Manish Gulyani, who said, “IP used to be the sexy thing, and optical was just fine. I think it's kind of reversed. Nothing new is happening in IP, and in optical, every day there's some new, cool innovation. With AI, optical is on fire.”

However, the size of that fire will be tempered by financial fuel.

“All that compute is generating so much data that will now need to be consumed. It will need to leave the hyperscale data centers to be consumable for enterprises and consumers alike,” Hatheier says.

“Training costs money, but inference and reasoning are what ultimately will generate money for the hyperscalers.”

Rob Shore, who previously worked at Infinera and is now part of portfolio marketing for optical networks at Nokia, agrees, noting that this balance will require further innovation.

“I think there's a whole new breed of optics that are going to be needed,” Shore says. “In addition to having trouble with the power grids, data center operators are having trouble fitting everything into a single building, so what they're having to do is break up a single data center into a bunch of smaller data centers that they want to act like a single data center, but those could be as far as 20 km apart. So, how do I interconnect those with low-cost optics, low-latency optics?”

And it’s not just cost. Matt Rehder, VP of core networking at AWS, explains that the hyperscaler is really more interested in stable connectivity rather than the highest performing connection to support the initial rush around AI and generative AI (genAI)

“A lot of the genAI workloads are just more sensitive to failures,” he says. “They're running at higher tolerances, and we have

protocols, we have everything else in place to make the network as resilient as possible and react to failures.

“But if you can fundamentally have your network fail less because your devices and links are just significantly more reliable, it's a much easier foundation to build on.”

Network operators sense a networking opportunity

This AI-infused opportunity is driving network operators to re-evaluate their own business models that have traditionally been mostly focused on basic connectivity.

Lumen Technologies, for instance, is betting big on an AI-infused networking future that will demand significantly higher throughput and lower latency than what is being provided today and will drive what CTO Dave Ward previously described as “the largest expansion of the Internet in our lifetime.”

Lumen’s investments are geared toward supporting what the vendor said is a standing $5 billion in aggregate sales it has already booked and the opportunity for an additional $7bn in sales pipeline.

The company’s CEO Kate Johnson told investors during an earnings call last year that the $5bn in aggregate sales is from “hyperscalers, it’s social platforms, it’s huge technology companies, it’s a cloud company.”

Johnson said she expects the next “tranche of demand” to come from the “AI model inference phase, probably with forward-thinking enterprises who see AI as a way to transform their businesses. Think

financial services, health care, and retailers to start.”

Ward explained that this AI inference push will also spill over into hybrid architectures that will see data held onpremises and using inference models associated with trained models in the cloud. “Massive amounts of bandwidth and super tight latency requirements for that,” Ward said.

“What this means for us as a connectivity partner is we fully plan on building an AI fabric between these locations and major data centers that are of the right power, size, scale, to host GPUs and the AI workloads and create a specialized connectivity fabric just for that purpose,” Ward said of this effort. “So much fiber and so many waves are required, and that segment has really become a different segment than some of the other cloud economic segments that are coming in that we can construct just to that.”

Telecom operators are also in a position to take advantage of this AI-related networking surge due to their diverse geographic footprint.

Speaking to SDxCentral, DCD’s sister publication, Juniper Networks CEO Rami Rahim said that the inferencing opportunity can be targeted at telecom operators that control prime real estate within the AI ecosystem.

“Connectivity alone is not enough to pay the bills so one of the most important emerging models is the understanding that inferencing is not all going to happen in centralized locations. It's, in fact, going to

happen closer to where the data resides,” Rahim explained. “In some cases, it might make sense for large enterprise, large financial organizations, and banks to do that inferencing on-premises. But I also think there's a very large opportunity for telcos to do that inferencing where they have beach-front property at the edge that connects directly to their enterprise customers.”

Telecom CEOs have been discussing this opportunity.

Verizon recently signed a deal with Nvidia to power enterprise AI services and digital transformation efforts running over the carrier’s 5G private network and mobile Edge compute (MEC) infrastructure. Adam Koeppe, SVP of technology planning at Verizon, touted the carrier’s architecture and its ability to support AI-derived use cases.

“I think it’s really important to look at what these capabilities exist within the architecture, and how can AI, true AI, augment things that are already being done or create things that are brand new,” Koeppe said in an interview with SDxCentral. “Where I see our evolution occurring is when you have an advanced cloud platform, as we do, you have an orchestration layer on top that we already have, and you then find ways to incorporate new AI capabilities on top of that. That’s going to allow your engineers and your operators to interface differently.”

T-Mobile US is also working to turn its cell site locations into Edge data centers or cloud connections that can in turn better serve AI workloads.

“These kinds of AI workloads will increasingly demand that processing happen on the device, which we’re starting to see, but there are obvious limitations there, or on a cloud near the device, and that’s a business opportunity that we see in the future,” T-Mobile US CEO Mike Sievert said.

Cisco’s Gartner noted that these opportunities are set to drive further investments.

“AI could be a stimulant for service provider customers that either begin to host AI applications for enterprise customers or as the enterprise market starts to build out [on-premises] solutions,” Gartner said. “We can start to see service providers having to build out more capacity as well.” 

Turn static files into dynamic content formats.

Create a flipbook
The AI Hardware Supplement May 2025 by DCD Magazine - Issuu