Diffusion language models' any order generation skips logical pivot tokens, hurting math and code reasoning. A left to right training constraint plus standard reinforcement learning recovers most of the accuracy.
The Alibaba-Tsinghua team that won ICML 2026 Outstanding Paper found that diffusion language models' any-order generation hurts their reasoning on math and code. A standard reinforcement learning technique applied after a left-to-right training constraint closes most of the gap, without slowing inference.
Diffusion language models, or dLLMs, generate text by iteratively refining a noisy canvas, much like image diffusion models, instead of writing token-by-token left-to-right. The flexibility has powered recent releases from Google's Gemini Diffusion to the open LLaDA model, with arbitrary-order generation pitched as a speed and creativity advantage over transformer LLMs. The team, working through the Tsinghua LeapLab, argues that flexibility carries a hidden cost on tasks where reasoning matters.
The paper, "The Flexibility Trap", names the cost "entropy degradation." During arbitrary-order generation, high-uncertainty logical pivot tokens such as "therefore," "so," or "hence" get skipped over because surrounding context is not yet fixed. Once adjacent tokens are filled in, those pivot words become fill-in-the-blank rather than load-bearing reasoning steps. The output reads fluently while the reasoning chain drops out of the generation.
The empirical asymmetry is large, per the paper's reported results. On HumanEval, a standard code-generation benchmark, 21.3% of problems are solvable left-to-right but not under arbitrary order. Only 0.6% are solvable under arbitrary order but not left-to-right. The asymmetry runs roughly 35-to-1 in favor of sequential reasoning, and the same pattern appears on GSM8K, the grade-school math benchmark.
The proposed fix is called JustGRPO. GRPO is a standard reinforcement learning technique used in left-to-right model training. JustGRPO applies the same objective but forces a left-to-right generation order during training, abandoning the arbitrary-order workarounds that earlier diffusion-RL algorithms tried to preserve. On GSM8K, JustGRPO reaches 89.1% accuracy, surpassing specialized diffusion-RL algorithms d1, ESPO, SPG, and GDPO. The project page documents the full results and hosts the code.
The result recasts a defining trait of the dLLM category as a design liability for math and code, where logical structure is load-bearing. Diffusion language models have drawn interest partly because vendors have claimed faster parallel decoding and richer exploration of solution paths; the paper argues the second claim carries a steep cost on any task that requires a chain of inference. With a sequential training constraint, dLLMs can use the same RL machinery that has powered the post-training of frontier LLMs. Inference latency stays unchanged because the constraint applies only at training time.
ICML 2026 named a second Outstanding Paper on diffusion language models, from Google DeepMind, confirming the conference's view that dLLMs are a live research frontier. Coverage of the Alibaba-Tsinghua award so far has come mainly from Chinese tech press, including 量子位, 163.com, and Sina Finance. Whether major dLLM providers adopt the left-to-right constraint, or double down on any-order generation, will determine how quickly the trade-off the paper names shows up in production systems. The code is open-sourced on GitHub for replication.