AI Clusters Are Growing Faster Than the Wires Connecting Them

The next jump in AI cluster size is being throttled by something that has nothing to do with transistors: the wiring between racks.

The largest training builds have crossed from a few thousand chips into 10,000- and 100,000-card configurations. The distance between racks now stretches from a couple of meters to 100 meters, and in some designs to 200 meters. The cables that connect them have become the binding constraint on whether that scale is physically possible.

The pressure shows up in a number. Frontier-model training compute is growing roughly 4-5x per year, according to EPOCH's published research. That is the demand side. The supply side is the wiring inside the data center, and the wiring is not keeping up on the same curve.

The industry's response is a three-way bet on how to move data using light instead of electricity across those longer distances. Each option trades off power, latency, manufacturability, and how tightly the optical components are tied to the switching chip.

The first approach, Digital Pluggable Optics (DPO), is the established workhorse. A pluggable optical module sits in the front panel of the switch, converting electrical signals to light at the rack edge. DPO is mature, the supply chain is broad, and operators can swap modules without redesigning the switch. The cost is power. Every conversion between electricity and light burns energy, and the retimer chips that clean up the signal draw more.

The second, Co-Packaged Optics (CPO), moves the optical engine onto the same substrate as the switching chip. The signal stays in optical form for a shorter distance, which cuts power and latency. The trade is manufacturing and service. Advanced chip-level packaging raises the cost of every unit, and replacing a failed optical engine becomes a board-level repair rather than a swap. An industry explainer pegs mainstream CPO production as a 2028-or-later timeline, and SemiAnalysis's CPO scaling note treats the same window as a ramp rather than a clean launch. A separate market research forecast for CPO places 2027 at the start of that ramp.

The third, Near-Package Optics (NPO), puts the optical engine on the edge of the chip package substrate, roughly 5 centimeters from the ASIC. It removes the retimer without forcing the optics and the switch into the same package. The result keeps standardized interfaces and a fully pluggable supply chain, which the same industry explainer positions as the practical middle ground between DPO and CPO, in part because it lets the two evolve separately.

AMD, Broadcom, and NVIDIA have joined hyperscalers Meta, Microsoft, and OpenAI to define an optical scale-up interconnect for AI clusters, with the eventual target of 3.2 TB/s per link. The presence of all three switch vendors in the same effort is unusual. It suggests the industry is treating the interconnect as common infrastructure rather than a competitive moat.

None of the three options is a clean win. DPO keeps the supply chain flexible but pays a power tax. CPO is the most efficient on paper but the hardest to manufacture and service. NPO sits in between on every axis and is the option that lets operators keep the most existing tooling.

There is a clean test for whether this framing holds. If inter-rack bandwidth keeps scaling in step with cluster size, the bottleneck stays on the chip side, and the wiring argument collapses. The current signal, with cluster sizes having grown over 20x from 2016 to 2024 while per-rack compute keeps its own curve, is what makes the wiring case load-bearing rather than speculative.

The next generation of AI infrastructure will be decided, in part, by which of these three optics bets the largest operators are willing to standardize on first.