The benchmarks that would make AI research autonomous are already being solved. What happens after that?
The benchmarks that would make AI research autonomous are already being solved. What happens after that?
Jack Clark has been writing about AI for long enough to know what a reckless prediction sounds like. That makes his latest one notable.
In a newsletter published Monday, Clark, the co-founder of Anthropic and former head of policy at the lab, assigned a 60 percent or better probability to a specific outcome: that AI systems will be conducting AI research with no human involvement by the end of 2028. Not assisting. Not augmenting. Replacing.
"It's a reluctant view because the implications are so large that I feel dwarfed by them," Clark wrote. "I'm not sure society is ready for the kinds of changes implied by achieving automated AI R&D."
The prediction would be easy to dismiss as another in a long line of AI timelines from the anxiety-producing corner of the AI world. But Clark's essay is different in one important way: it's built almost entirely on public benchmark data, and that data is striking.
Start with SWE-Bench, a coding test that asks AI systems to resolve real GitHub issues. When it launched in late 2023, the best score was roughly 2 percent. Claude Mythos Preview, released in April, scores 93.9 percent. The benchmark is effectively saturated.
Or consider CORE-Bench, which tests whether an AI agent can reproduce the results of a scientific paper from its code repository. In September 2024, the best system scored about 21.5 percent. By December 2025, one of the benchmark's own authors declared it solved, with a model achieving 95.5 percent.
The speedup numbers are similarly stark. Anthropic has been tracking how well its models optimize LLM training code. In May 2025, Claude Opus 4 achieved a 2.9x average speedup. By April 2026, Claude Mythos Preview hit 52x. For context, a human researcher typically needs four to eight hours to achieve a 4x speedup on the same task.
METR, an independent research organization that measures how long a skilled human would take to do what an AI agent can do, tracks the same trend. In 2022, GPT-3.5 could handle tasks taking a human about 30 seconds. By 2026, that has risen to roughly 12 hours with Opus 4.6. Ajeya Cotra, a longtime AI forecaster at METR, projects roughly 100 hours by the end of this year.
This is the evidence mosaic Clark is looking at. "All the pieces are in place for automating the production of today's AI systems," he writes. "The engineering components of AI development."
The question that follows is where it gets interesting.
Clark doesn't argue that AI can already do everything a human researcher does. He explicitly sidesteps the question of whether current systems can generate the kind of radical, left-field insight that occasionally reshapes a field. What he argues is that the engineering work — the actual implementation, the iteration, the debugging, the optimization — is already within reach. And if scaling trends hold, the creativity layer may not be a permanent wall. It may just be another engineering problem waiting to be solved.
The timeline convergence is worth dwelling on. Daniel Kokotajlo, an AI researcher who maintains probabilistic forecasts on LessWrong, revised his own estimate for when an AI will be able to replace human software engineers — his "Automated Coder" milestone — from late 2029 to mid-2028 in a Q1 2026 update. He cited METR's updated data and the rapid improvement of agentic coding tools. Sam Altman, the CEO of OpenAI, said in January that having an automated AI researcher by March 2028 was "an internal goal" of his company. He added: "We may totally fail at this goal."
The disagreement among serious people isn't whether progress is happening. It's whether the remaining gap is categorical or merely engineering.
There is a version of this story that focuses on timelines. Clark says 2028, Altman hedges, Kokotajlo agrees with the direction if not the exact date. That's a real and legitimate news hook. But there's a deeper question underneath it, and it's one that Clark's essay touches without fully naming.
The question is not whether AI will automate research. The question is what happens to the idea of research itself when the system selecting which problems to solve has no connection to human desire, fear, or curiosity.
Research has always been a human-directed activity. Not just in the sense that humans run the experiments, but in the sense that human values — what we find interesting, what we think matters, what risks we're willing to take — determine which questions get pursued. A scientist follows a hunch. A field develops intuitions about which problems are worth solving. Serendipity plays a role. So does career incentive, institutional pressure, cultural context.
When an AI selects which problems to solve, none of that applies. The selector is optimizing against a metric, not a meaning. This isn't a philosophical abstraction. It has a concrete implication: the output of automated research could be a stream of genuine discoveries that nobody particularly wanted, arranged in a priority order that reflects nothing human.
Clark frames his essay as a progress report, not a prophecy. The benchmarks he cites are real. The trends they show are real. Whether they lead to full automation by 2028 or 2032 or never is genuinely uncertain. Altman says it might not happen at all.
But the direction of travel is not uncertain. The engineering components are being crossed in real time. What happens after — who decides what gets built, for whom, toward what end — is a question that the benchmarks can't answer. And that question is worth asking before the answer arrives.