Why the CPU is doing more work in your AI agent than you think

The chatbot era treated the GPU as the whole story. A language model is mostly dense matrix math, and GPUs are built to do that math very fast, so a 1:8 CPU-to-GPU ratio became the default rule of thumb for LLM serving. When an AI can call tools, run code, and chain multiple steps, that ratio starts to crack.

In an agentic workflow, the model emits a token, the host CPU dispatches a tool call, the tool returns data, and the model thinks again. Each step is sequential and conditional: the next move depends on the previous result. That work is not matrix multiplication, and the GPU spends much of it waiting. The Red Hat blog summarizing Intel's argument puts the new ratio at 1:1, and in some agentic deployments as tight as 4:1 CPU-to-GPU. Treat those numbers as Intel-originated vendor claims. The closest independent academic-style support is the arXiv preprint from Georgia Tech and Intel, which frames agentic serving as CPU-centric and analyzes orchestration overhead on the host. It is not a third-party benchmark.

The mechanism is latency, not FLOPS. GPUs optimize for throughput on large batches, where they can amortize memory traffic across thousands of similar operations. They are poor at the small, branchy, I/O-bound work that dominates an agent loop: parsing a tool schema, launching a subprocess, waiting on a network call, or reading from a context window of millions of tokens. CPUs were built for that. The Hacker News thread on the Red Hat post caught the tension: several commenters called the post "AI slop," while agreeing directionally that the CPU share is rising in agentic stacks. That is the right read. The vendor framing is loud, but the underlying workload shift is real.

Viksnewsletter's analysis of the CPU bottleneck in agentic AI lays out the practical effect: in production agentic serving, tool-dispatch latency and context-handling overhead tend to dominate end-to-end "time to first useful token" far more than raw inference speed. In other words, you can buy the fastest GPU on the market and still ship a slow agent if the host CPU is the choke point.

The vendor context is not incidental. Intel's Computex 2026 keynote made the CPU-for-agentic argument a centerpiece of its commercial narrative, and the company's Q1 2026 results, referenced in the same newsroom coverage, reflect a push to make that argument pay. So when you read a "the CPU is back" headline, you are reading an Intel-shaped message, and the specific ratio numbers should be treated the way you would treat any vendor benchmark: as a directional claim until something independent reproduces them.

For builders, the practical move is to stop measuring agentic latency in tokens per second on the GPU and start measuring it end-to-end across the host. Profile where the wall-clock time actually goes in a real agent trace. If the tool-dispatch step is 80% of the latency, a faster GPU will not help. If context assembly is the bottleneck, more CPU cores and more memory bandwidth will. The workload placement question is not "GPU vs CPU." It is "which step is sequential and which is parallel, and is the hardware matched to the step."

The cheap prediction is that the GPU-only consensus will keep softening as more agentic products ship, and the CPU:GPU ratio in production stacks will drift toward parity for inference-heavy agentic workloads while staying GPU-heavy for training. The harder prediction is whether the specific 1:1 or 4:1 numbers will hold up under independent benchmarks, or whether the underlying mechanism (sequential orchestration as the gating constraint) is the durable finding and the ratios are just the vendor's first draft.

That distinction is the one to watch.