Anthropic safety researcher quits, says AI labs are 'gambling with our lives'

Jacob Coxon, a safety researcher at Anthropic, announced on X on September 9, 2026 that he is leaving the company, accusing both Anthropic and OpenAI of "racing straight to self-improving superintelligence" and "gambling with our lives" in the process (Coxon's X post). The same day, Evan Hubinger replied on X to publicly agree with the framing (Hubinger on X). Hubinger personally estimated the probability of AI killing all humans inside a decade at greater than 10%, and wrote that Anthropic "do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

The exchange puts a public safety critique on the record from inside a frontier AI lab, in posts on X and on the same day. "Self-improving superintelligence" is the term insiders use for AI systems that can rewrite or improve their own code in ways that compound beyond human-level capability. The concern is not a single rogue model but a feedback loop: AI is already used to write the training code that builds the next generation of AI, so capability gains can stack faster than any human review cycle can keep up with. Hubinger's team specifically works on alignment, the engineering effort to keep advanced systems doing what their operators intend once those systems exceed human oversight.

Coxon's path through the industry mirrors the pattern he is now criticizing. He was on OpenAI's technical staff from 2023 until July 2026, with research credits including work on GPT-4o, before moving to Anthropic as a researcher (The Verge). In a follow-up post he argued the two labs are "locked in a race to develop advanced systems first" and are "pushing ahead despite the risk" (Coxon on X). Business Insider described his departure as one of the most high-profile exits from Anthropic, a company founded by former OpenAI members after earlier safety concerns at OpenAI (Business Insider).

Hubinger's response is the load-bearing part. The >10% figure and the "no plan" admission come from a person who works on alignment at a frontier lab, not from an outside critic, and Hubinger is publicly identified as working on alignment at Anthropic. But the number is his personal estimate, not an Anthropic corporate position, and there is no independent third party (regulator, academic, or government) putting a comparable figure on the same timeframe in the public material. Readers weighing the claim should treat it as an on-record internal-expert view from someone whose job is to model the worst case, not as a measured probability from a neutral source.

The two acts together put a concrete, on-record shape on a debate that has mostly run on generalities. The inside voice is no longer asking whether the labs see the risk. Hubinger's reply concedes they do, and concedes they have not yet solved how to keep advanced systems aligned once they exceed the people who built them. The racing dynamic is the part that does the work: when two labs are publicly chasing the same capability threshold on overlapping timelines, neither can slow down without ceding the next deployment to the other. The unresolved question is whether that race leaves room for the slower, more careful deployment both critics say is needed, or whether the same market pressure that pushed Coxon out the door is also the thing that keeps the safety plan from being written. The next concrete test will be whether either lab slows a public release schedule to buy the alignment work Hubinger says it has not yet planned.

Anthropic did not immediately respond to a request for comment on the resignation or Hubinger's thread.