The verifier is now the system

Automation is starting to make yes/no calls in the parts of the economy where wrong answers cost real money. The new bottleneck is not the model that reasons. It is the auditor that decides whether the reasoning was any good.

Krasser and Raff's CrowdStrike paper on security alert triage made the case with a clean counterfactual. A reasoning-enabled model hit 82.6% accuracy on real Windows endpoint alerts. The calibrator that audited the model's own chain of thought bumped benign recall by 43.0% and malicious recall by 18.3% at the operating point that gates automated triage. Remove the calibrator and high-confidence recall collapses to zero. The skeptic was not a flourish. It was the system.

Anywhere a language model is being pushed from suggestion to verdict, the confidence score that automation depends on is being scrambled by the act of reasoning out loud. The bar for "deployable" is no longer accuracy. It is whether a separate model is watching the reasoning and saying yes before the call goes out.

The honest caveat: this is one vendor's data and not yet peer reviewed, so the calibrator win could be specific to this distribution. The leading explanation is structural, since chain-of-thought reasoning is known to shift output distributions in ways a direct-label classifier never had to absorb. The next unit of work is the verifier, and most teams building on language models are not building it yet.