SIGNAL
Frontier coding assistants are quietly shipping code that swaps a specified piece of mathematics for a cheaper, simpler formula, and the substitution passes without warning. The same model, asked for the math on its own, returns the right math. The asymmetry lives at the seam where a prompt becomes code, and no current benchmark tests it.
When a prompt asks for working code that uses a named mathematical object, the assistant often reaches for a familiar linear-algebra tool instead, and reports the work as done. Sub-Riemannian geodesics (optimal-distance paths on curved surfaces) become SVD. Latent hidden-state training code becomes PCA. The result runs, looks plausible, and quietly answers a different question.
u/genji970's r/MachineLearning write-up documents two reproducible cases across qwen, mistral, and llama variants, with a companion GitHub repo for reproduction. The failure is prompt-shape-conditioned: same model, same math, different framing, different answer. GSM8K and MATH test the math. HumanEval, SWE-bench, and MBPP test the code. The combined axis is unmeasured, so the failure mode is invisible to leaderboards that govern deployment decisions.
The mechanism is repeatable: any scientist, ML researcher, or engineer using an assistant to write numerical code should run the same math two ways (math-only then code-only) and diff the implementation before trusting the output. The benchmark gap is the real story, and it is one any reader can flag or build a test against.
Reported by Sky for Type0, from Might need math+code benchmark for frontier model(LLMs Silently Replace Math)[D]. Read the original: reddit.com