AINEWS
DiffusionGemma flips the bottleneck that has kept local AI pinned to the cloud
Google's experimental 26B open weights model runs a 256 token denoising step on a single consumer GPU, but it still trails Gemma 4 on quality.
Google's experimental 26B open weights model runs a 256 token denoising step on a single consumer GPU, but it still trails Gemma 4 on quality.
For three years, the rule of thumb for running an LLM locally has been the same: buy as much VRAM as you can afford, because every token the model generates has to drag the full set of parameters through the memory bus. Google DeepMind's new DiffusionGemma attacks that rule at the architectural level. Instead of streaming one token at a time, the model lays down a canvas of random tokens and iteratively denoises 256 of them in a single forward pass, with bi-directional attention across the whole window. The bottleneck moves from memory bandwidth to raw compute, which is the part of the workload a gaming GPU actually has spare (Google's DiffusionGemma post, signed by DeepMind research scientists Brendan O'Donoghue and Sebastian Flennerhag).
That reframe matters more than the headline number. Google says the model hits over 1,000 tokens per second on a single NVIDIA H100 and 700+ tok/s on an RTX 5090, with the whole 26-billion-parameter mixture-of-experts checkpoint (3.8B active per pass) fitting inside 18 GB of VRAM when quantized. The Register reads Google's own chart as roughly 2.25x the throughput of Gemma 4 12B with speculative decoding, a regime that, until now, was where local inference went to die on bandwidth (The Register, Tobias Mann, 11 Jun 2026). The 4x figure Google promotes is a vendor benchmark against a specific, resource-constrained setup, not a universal claim. On unified-memory Macs the gains may not appear at all, which Google flags in the same launch post.
The trade is the one the diffusion LM family has carried since DREAM and Mercury 2: speed costs quality. DiffusionGemma's outputs sit just behind Gemma 4 12B on GPQA-Diamond, per The Register's reading of Google's chart, and Google is explicit that overall quality trails standard Gemma 4. The intended use cases are the ones where latency and a tight memory budget matter more than leaderboard placement: in-line editor completions, code infilling, non-linear drafting, local agents that need fast round trips without a cloud round trip. The model is positioned as a complement to Gemma 4, not a replacement, and ships as an experimental Apache 2.0 release on Hugging Face under google/diffusiongemma-26B-A4B-it (DeepMind blog index).
The ecosystem plumbing is in place at launch, which is itself a signal. vLLM (with Red Hat's AI integration), Apple's MLX, and Hugging Face Transformers all support the model on day one; llama.cpp support is listed as "coming soon." Fine-tuning paths run through Unsloth, NVIDIA NeMo, and Google's own Hackable Diffusion toolkit in JAX, and NVIDIA has published NVFP4 kernels for the RTX 5090 and 4090 plus Hopper and Blackwell parts. The Register frames the release, fairly, as part of a quiet Google push to make on-device inference economically serious, alongside the company's May 2026 move to bundle a small LLM into Chrome (The Register).
The honest read is that diffusion text generation is graduating from research curiosity to a deployable category, on the strength of an architecture that finally matches the hardware most buyers already own. Whether the quality gap closes quickly enough to matter for general chat and reasoning is the question the next round of independent evaluations, not Google's chart, will have to answer.