Your GPUs aren't slow. Your model is the wrong shape.

A team spent several million dollars on eight H100 GPUs (NVIDIA's previous-generation flagship AI accelerator) for a recommendation system, watched utilization hover near 40%, and reached for the procurement card. NVIDIA's latest technical blog, "AI Model Co-Design: Hardware-Friendly LLM Design," says put the card away. The chip is fine. The model is the bottleneck.

That framing, drawn from a Leiphone explainer of the post, is the entire argument. When GPUs sit underused, the usual story is that someone needs to buy more of them. NVIDIA's engineers argue the opposite: most of the waste is upstream, in the shape of the matrix the model multiplies.

The first tool is arithmetic intensity, a single number: compute operations divided by bytes moved. When that ratio is high, the math units on the chip stay busy. When it is low, they wait on memory. A layer that is "memory-bound" cannot use a faster GPU, because the chip is starved for data rather than for compute. A layer that is "compute-bound" is the one where adding more silicon actually helps.

A worked example anchors the difference. On the GB300 (the current flagship AI chip in NVIDIA's Blackwell generation), the silicon offers roughly 15 PFLOPS of FP4 throughput and 8 TB/s of memory bandwidth. Take a standard layer in a transformer, the second projection inside the feed-forward block (what NVIDIA calls FFN-2), with the standard 8192 hidden width. If the inner dimension is K=512, the layer is memory-bandwidth-bound at every token count, no matter how many requests land in the batch. The math units are idle not because the chip is slow but because the matrix that needs multiplying is too skinny to keep them fed.

Push K past 3072 and throughput clears 80% of the chip's peak. Push it to 6144 and the chip is fully fed. The shape that starves the silicon and the shape that feeds it differ by a factor of 12. The same arithmetic intensity lens applies to almost every layer in a large model, and the same penalty applies whenever that ratio is wrong.

If you fix the shape but switch precision, the gap gets wider. NVFP4, NVIDIA's native 4-bit floating point format on Blackwell, accelerates compute faster than it accelerates memory, so memory-bound layers become relatively worse, not better. A team's good-news precision upgrade can quietly make their underutilization problem more obvious, not less. The technical case is laid out in "3 ways NVFP4 accelerates AI training and inference," and NVIDIA's own DeepSeek-R1-NVFP4 release on Hugging Face is the deployed artifact that proves the path runs end to end. A preprint from NVIDIA on NVFP4 pretraining makes the same point from the training side: a precision format is only as useful as the model shape it lands on.

NVIDIA's blog distills a co-design playbook into seven rules, ordered by throughput impact. Skip the skinny matrices. Align every dimension to the chip's tile sizes (128, 256, 512), so the tensor cores run at their native granularity. Pick the numerics the hardware accelerates natively; mismatched formats burn cycles in software. Pre-shard the model along the actual network topology so all-reduce traffic matches the bandwidth available between GPUs. Treat the throughput-vs-latency trade-off as a Pareto curve, not a knob: the only sustainable move is to push the whole curve outward. And treat model design and hardware choice as a single coupled decision rather than two separate ones.

The rule about tile alignment is the cheapest win and the most easily missed. A dimension like 333, an odd, prime number that is not a multiple of the chip's tile, forces the kernel to pad the matrix to a usable size, waste compute on the padding, and ship a smaller fraction of the result. Choose 384 or 512 instead and the cost disappears.

NVIDIA sells the chips and the software stack that makes them useful, so the company has a commercial interest in locating the inefficiency in the model rather than in the hardware. That does not make the argument wrong. The arithmetic intensity math is independent of who wrote the blog, and the GB300 measurements are reproducible on the chip itself. But the framing is one a reader should hold with the source's position in mind: when the chip vendor says you don't need more chips, the next question is what shape your model is.

The next time someone proposes a new GPU order, the first question is whether the current cluster is being fed. If utilization is stuck and the math says the layer is memory-bound, more silicon does not close that gap. The same diagnostic, arithmetic intensity, memory-bound or compute-bound, tile alignment, and NVFP4 fit, travels into every future AI-efficiency decision, including the next datacenter RFP.