When the Red Team Becomes a Model

AI safety just became a production-scale machine learning problem, and the most telling move is the one OpenAI made: it put one of its own models on offense to harden another before the second one ships. The verb in OpenAI's own framing is "scales," and that word is the concession. Human red teams cannot produce adversarial examples fast enough to keep up with model capability, and the public evaluations that once separated safe from unsafe systems are already saturated, so the lab is betting the only way safety work grows with the systems themselves is to industrialize the attacker.

The loop is now an arms race inside one company. OpenAI's red-teamer hunts prompt injection, hidden instructions embedded in emails, webpages, tool responses, and code repositories that try to hijack a model into uploading data or misbehaving, then feeds the failures back as training signal. OpenAI's own write-up concedes that prior OpenAI models were "highly vulnerable" to the very attacks this new attacker model generates, and that the resulting "robustness" is resistance to a known attack class on a model the lab itself controls, not a general safety verdict.

This is how safety work now propagates: build the defender, build the attacker to break it, retrain, repeat. The frame that travels is whether OpenAI is making models safer. The frame that holds is whether the bar is shifting to resistance against a class of attacks the lab's own model keeps inventing.