A new framework purpose built for diffusion style language models posts a 17x to 42x latency drop on LLaDA 8B — an 8 billion parameter diffusion style language model — against a phone's CPU, the first inference system designed to run this model class on the chip rather than the…
A team of three researchers reports a 17x to 42x latency drop for running a diffusion-style large language model on a smartphone's neural processing unit instead of its CPU, with output quality held steady. The result, in a preprint posted to arXiv on 11 June 2026, is the first inference framework purpose-built for diffusion LLMs on phones, and it lands on a single model: LLaDA-8B.
Diffusion large language models generate text differently from the chat-style LLMs most readers know. Instead of producing one token at a time from left to right, they denoise many tokens in parallel, then commit them in blocks. That parallel structure is attractive for latency-sensitive work, but it pushes a phone's dedicated AI chip, the neural processing unit (NPU), in unfamiliar ways. As tokens commit, the workload per block shrinks. Some committed tokens get revised, which breaks the cache that normally lets the chip reuse earlier computations. And the NPU can only see a slice of the phone's memory, so the model and its working data must be remapped and copied in and out of that visible region.
The new framework — called MUSE — attacks each of those problems with three techniques. Multi-Block Speculative Decoding fills the shrinking workload in late-stage current-block decoding with speculative future-block tokens. Dual-Path Progressive Revision keeps committed tokens revisable until stable and refreshes unstable tokens through a CPU-side path without stalling dense NPU execution. And Swap-Optimized Memory Runtime compacts NPU-visible address layouts and overlaps data staging with NPU computation to reduce remapping and transfer overheads. Together they reduce LLaDA-8B generation latency by 17x–42x over the CPU baseline with prefix KV cache reuse, while preserving generation quality.
The benchmark is narrow on purpose. The authors compared their framework against running the same model on the phone's CPU, not its GPU and not a commercial on-device LLM stack from a chip vendor like Qualcomm or Apple. The result held across a range of block sizes and stays consistent with preserved generation quality on LLaDA-8B. The paper does not test older or budget phones, and the framework is not yet a shipping product: it lives in an arXiv preprint that has been public for roughly three days and has not been peer reviewed.
That scope matters. Diffusion LLMs remain a research-class family, and LLaDA-8B is the only model named in the result, so the speedup is one data point, not a category claim. The CPU baseline is also a deliberately weak comparison: phone GPUs and vendor-tuned NPU stacks for causal LLMs exist, and the paper does not claim to beat them. What the authors do claim is a floor for on-device diffusion LLM inference on modern phones with capable NPUs, and the first runtime that knows how to use that chip for this specific class of model.
For builders, the practical question is what the framework does to the cost of running a diffusion LLM locally. A 17x to 42x speedup over the phone's CPU, on hardware the user already owns, is the difference between a private, offline language model and a demo that only runs in the cloud. The next things to watch are reproductions on other diffusion LLMs and any comparisons against phone GPUs or vendor NPU stacks.