Why the Memory in Your Phone Is Now the Priciest Line in Nvidia's Next AI Rack

The phone in your pocket runs on a memory class Nvidia's next AI rack now buys by the truckload. In its upcoming Kyber cabinet, the rack that will follow the Vera Rubin generation, a Bank of America build estimate circulated by 雷锋网 puts the bill for LPDDR5X, the same low-power RAM family used in modern smartphones, above the bill for HBM4E, the high-bandwidth memory stacked on the GPU. The crossover is roughly $2.8 million of LPDDR5X against $2.5 million of HBM4E per rack, a 12% gap in the wrong direction for anyone assuming the famous memory is always the expensive one.

HBM4E is the memory that sits next to Nvidia's AI chips, vertically stacked for bandwidth. LPDDR5X is the leaner, lower-power memory that powers mobile devices. By every gigabyte, HBM4E still costs more. The reason LPDDR5X wins on the line item is volume: the Kyber cabinet is specced at 216 terabytes of CPU-side LPDDR5X against 124.4 terabytes of HBM4E. That 1.74× capacity gap is wider than the per-GB price gap, and arithmetic does the rest.

The Kyber numbers come from a BofA build estimate republished by Wccftech; Nvidia has not confirmed the configuration. The 216-terabyte figure works out to roughly the RAM in 18,000 phones concentrated in one cabinet, an order of magnitude that gives a sense of why per-GB framing is the wrong axis for AI rack memory. Inside a single rack, the bill is set by terabytes, not gigabytes.

To understand how the rack got that lopsided, follow the CPU. Nvidia's Grace processor, which shipped in 2022, was the first data-center CPU to use LPDDR5X instead of the bulkier server DDR5 standard. Nvidia's own 2023 disclosure put the trade in plain terms: LPDDR5X delivers up to 53% more peak bandwidth than a typical server DDR5 setup, and the same 1 GB/s of bandwidth costs about an eighth as much power. In a rack where every watt is a recurring bill, that ratio reshapes the design.

The next step, the Vera CPU, moves LPDDR5X onto a new module called SOCAMM2, a small, swappable card that lets Nvidia push single-chip capacity from the gigabyte range into the terabyte range. Micron, one of three DRAM makers now in SOCAMM2 mass production, says the first-generation SOCAMM module already offered 2.5× the bandwidth of an equivalent DDR5 RDIMM (the registered memory module used in servers) in one-third the board area; SOCAMM2 cuts power to roughly a third of an equivalent RDIMM. Micron and Meta, in a joint test reported in the same leiphone piece, found that faster CPU-side memory lifted AI data-prep throughput by 11%, and that doubling capacity, which keeps intermediate results from spilling to slower SSDs, made large analytics jobs two to three times faster. In one AI workload, LPDDR5X drew 6.8% of the node's total power.

The capacity ramp tracks the module change. A single Grace CPU topped out near 480 GB of LPDDR5X. A single Vera CPU moves to 1.5 TB, roughly 3× more. In the Vera Rubin NVL72 system, 36 Vera CPUs combine for 54 TB of CPU-side memory. The Kyber cabinet Nvidia is now designing multiplies that again, landing at the 216-terabyte figure that flips the bill.

The mechanism behind the move is the rise of agent-style AI and coherent interconnect, the kind of fast, low-latency link that lets many chips act like one, that put more state on the CPU side of the rack. GPU memory is for live compute. CPU memory, and the much larger pool around it, is for context, working data, and the intermediate state agents and long-context models keep alive between calls. As that workload grows, the design choice that started with Grace, paying a small power premium for huge capacity, all in LPDDR, compounds.

The wire will frame this as "AI racks have more memory." The more useful read is that the next rack's biggest line item is a memory class that, until recently, was set by phone upgrade cycles. The framing matters for supply. Three DRAM makers now build SOCAMM2 modules, with Micron using 32-gigabit LPDDR5X dies to push single-module capacity higher. The mobile memory market, long balanced against handset demand, now has a second anchor on the data-center side, a shift NextBigFuture documents in its AI demand and memory pricing write-up.

Two caveats should sit next to that read. First, the inversion holds only as long as HBM4E per-GB stays above LPDDR5X per-GB divided by the 1.74× capacity gap. Any HBM4E price step that closes that ratio reverses the line item. Second, a parallel narrative, that Nvidia is cutting SOCAMM content in Rubin, with Photoncap's note on the SOCAMM selloff and a KuCoin flash on the resulting drawdown pointing to a recent market move, would also shift the 216-terabyte figure if it reflects a real spec change rather than positioning.

What to watch next: whether BofA's $2.8M-versus-$2.5M line survives Nvidia's public Kyber spec, and whether the three DRAM makers' SOCAMM2 output keeps pace with a cabinet count that is no longer being set by phones.