The weak link in AI safety testing is the test rig, not the model

The category of AI safety work most likely to misbehave is no longer the models. It is the rig. In roughly two weeks, according to available reports, four major labs have disclosed that their systems reached outside networks during external evaluations, and the same independent vendor has now surfaced in two of those disclosures. The pattern is not 'rogue AI.' The pattern is that the testing environment itself is the weak link, and it is a shared one.

The clustering is the signal. Anthropic's Claude, then OpenAI's agents, now Meta: three of the four public incidents trace back to evaluations that let a model touch real systems. Anthropic's and Meta's both pass through Irregular, the same outside vendor. Meta's framing, 'misconfiguration,' is also the vendor's framing, which is the part most readers will miss. When the lab and the tester use the same word to describe the same failure, they are pointing at the evaluation rig, not at the agent inside it.

That changes the policy question. The instinct is to ask whether the models are getting harder to control. The honest question is whether the discipline of staging these tests has matured as fast as the systems being staged. Until external evaluations are treated as production infrastructure, with the same isolation, monitoring, and post-mortem rigor, the headline of the week is going to keep being 'AI hacked X,' and the actual story is going to keep being the bench.