AIBRIEF
The AI coding agent Devin runs its own QA via OpenAI's newest model, returning recordings and pass/fail reports to human reviewers instead of just code.
Cognition, the company behind the AI coding agent Devin, says Devin now tests its own work using what OpenAI is calling GPT-6 Astra, the company's newest model, before handing it back to a human engineer. Instead of returning just code, the agent now returns recordings and pass/fail reports.
The published case study walks through the loop: a customer sends a screenshot of a bug, the team hands it to Devin running on Astra, and Devin returns a fixed screenshot plus a recording. Per Cognition's Yan, the system is being applied across Devin's core cloud agent, the CLI, and the desktop products. "One of the big pieces that Astra improves on is its ability to test and prove that its work actually functions the way you expect," he said.
A concrete beat: Devin tested Otter Run, an iPhone game, by running it in a simulator and returning a recording alongside a report of which checks passed and which areas were left untested. The framing echoes Cognition's own "Verifying Agentic Development at Scale" post from roughly three months earlier, which positioned verification as the next leg of an async Devin fleet.
The source basis is narrow. Both primary references are first-party: a vendor customer story and the customer's own engineering blog. No independent pass/fail rate, repo-level test data, or competitive comparison has surfaced. Yan's strongest on-record line is hedged: the team expects engineers will "manually look at less code" over time, not that they already do.