Most quantum computing research can't be reproduced, audit finds

A reproducibility audit of 127 recent quantum computing papers found that only 24.4% came with code that could be tested, and roughly two-thirds of that code failed to run in a clean environment. Lead analyst Wolfgang Mauerer of the Technical University of Applied Sciences Regensburg called the numbers "a rather negative result" compared to traditional computer science standards and said the situation "didn't really improve over time" since his team's 2021 baseline.

The paper, "Works on My QPU" by Köster and colleagues, uses a five-question framework: does a study ship code, document its environment, describe the hardware, explain how to run it, and actually execute? Manual review found only 24.4% passed the code test; of those, 64.5% failed to run. A broader automated sweep of 4,966 papers found a 26.8% code-availability rate without the execution step, and about a third of papers with code lacked machine-readable environment specifications.

New Scientist reports this is the first large-scale reproducibility audit in a heavily funded field whose commercial use cases remain unsettled. The constructive note: declarative environment tools, systems like Nix and devenv.sh that pin dependencies and hardware specs into reproducible setups, could close much of the gap if venues required them. The field now has a checklist. Whether it adopts one is open.