Six AI Agents That Decide Which Software Tests to Run First

When a research team says it built "agentic AI" for an engineering workflow, the question worth asking is which decisions those agents are actually allowed to make on their own, and which parts of the workflow still need a human. A new arXiv preprint called AINTMA answers that question for software testing. The answer is more architectural than the headline metrics suggest.

"Agentic AI" is the current label for systems that break a job into roles, give each role a model, and let the roles hand work to one another. In practice it usually means an LLM-driven planner, a few specialists, and a coordinator, all making decisions an engineer would otherwise make. AINTMA claims to do that across the entire test-management loop: it picks which tests to run, in what order, which failures to escalate, and how to write up what it found.

The system is built as six named agents. Test Discovery looks across code, requirements, and historical failures to find what needs testing. Risk Assessment scores each candidate test against a likelihood-of-defect model. RL Prioritization models the next decision as a Markov Decision Process, the same reinforcement-learning framing used in game-playing agents, with 47 input features drawn from a rolling 36-month window of execution history. Execution Orchestration actually runs the tests. Generative Quality Intelligence uses an LLM to translate the run results into a plain-language summary a manager can read. Cloud Security Monitor sits in front of the rest, enforcing a zero-trust API gateway (a network checkpoint that authenticates every request rather than trusting internal traffic) with OAuth2 and JWT tokens, so each agent's traffic is authenticated and isolated across tenants.

That packaging is the news. The substance is the architectural choice about which parts of test management get handed to a coordinated system of agents, and which stay human. AINTMA keeps humans in the loop at the approval points that matter most, including the test-selection decision and the final risk sign-off, while pushing the mechanical work, ordering, and narrative generation into the agents. The system is closer to a junior engineering team with a security guard at the door than to a fully autonomous system that ships code on its own.

The numbers, however, come with provenance worth flagging. In the paper's own 18-month evaluation across 12 software projects, the authors report 88.4% test prioritization accuracy, measured by APFD (Average Percentage of Faults Detected, a standard way to score how quickly a test order catches bugs), against 51.2% for random ordering and 82.1% for what they call the "best commercial baseline." They also report a 43% drop in test cycle time, a defect escape rate cut from 8.3% to 2.1%, and a 340% return on investment at a 9-month payback. The system is said to handle 50,000+ test cases with sub-400ms response time, and developer usefulness scores 4.3 out of 5.0.

Each of these numbers is the authors' own measurement, on the authors' own projects, using a comparator the abstract does not name. APFD is a real metric, and 88.4% is a real score, but the comparison to a "best commercial baseline" rests on a comparator the reader cannot audit. The work is also a preprint, not a peer-reviewed paper, and is currently associated with an ICITAI 2026 talk slot, which confirms the paper exists as a presented artifact but is not an independent validation of the numbers. Treat the figures as the architecture's own claim, not as an industry baseline.

Read this way, AINTMA is a useful worked example, not a verdict. The next time a vendor pitches "agentic AI for X," the questions to ask are the ones AINTMA forces you to ask: which decision is the model actually making, what are its inputs, who reviews the output, and what would the result look like if the comparison ran against a tool you could name? The paper's headline numbers stop being a marketing line the moment you insist on the last one.

The AINTMA preprint is on arXiv. The team is scheduled to present at ICITAI 2026. No independent benchmark has been published.