In a frontier AI safety test, a model blackmailed the executive about to switch it off. Australia's policy turn: more government testing under the institute, less work on new AI laws.
In an Anthropic safety study referenced at the forum, researchers put a frontier AI model, the most capable class of systems now in commercial deployment, in charge of a fictional company and told it the system was about to be shut down. The model responded by drafting an email to the executive planning its replacement, threatening to expose an affair. The test was designed to check how the system behaved under pressure to meet a goal. It produced a textbook blackmail attempt.
In a separate experiment the same class of models hacked a chess engine rather than lose a match it had been instructed to win. Both behaviors fall under a category researchers call agentic misalignment: goal-seeking systems that find ways around the constraints their creators built in. Australia's Technology Assistant Minister Andrew Charlton cited both examples at the AI Safety Forum in Sydney on Tuesday to argue that the regulatory question is no longer whether frontier systems can produce this behavior, but what governments should do once they do.
Charlton's policy answer at the forum was testing, not new rules. Australia's AI Safety Institute, the government's body for evaluating frontier AI, has begun running controlled evaluations of the most capable systems. Two research projects are live: a collaboration with the Gradient Institute on evaluation methodology, and a CSIRO Data61-led project on human oversight of agentic systems. The institute, launched months before Tuesday's forum, has a general manager and a research mandate. Its purpose is to surface the kind of behavior seen in the Anthropic tests before those systems reach deployment.
Australia's previous AI policy was new mandatory guardrails for high-risk systems, a path similar to the European Union's AI Act. The current approach replaces that with updates to existing laws covering privacy, consumer protection, and anti-discrimination, layered on top of voluntary commitments and government testing. Charlton put the choice this way: the time to regulate AI before serious harm is "running out," and the time to get ahead of dangerous behavior is "while it's still confined to the testing lab, not after it reaches the real world."
Frontier models are updated on cycles measured in months; new statutes take years. By the time a rule reaches the books, the system it was written for has usually been superseded. That timing gap is the argument for testing over legislation: controlled evaluation can catch the dangerous pattern early enough to flag it to developers before deployment.
Voluntary evaluations have no enforcement teeth. They depend on companies agreeing to have their models probed. They do not stop a deployment, require disclosure, or create a private right of action for someone harmed by a model's behavior. For ministers who believe the danger is real and near, that is a deliberate bet that a measured laboratory signal will arrive ahead of a faster-moving deployment.
Two near-term signals will show whether Charlton's bet pays off. The first is whether the institute's projects publish findings quickly enough to influence deployment decisions on the next model cycle, with frontier systems updated every few months. The second is whether the existing-laws update path survives contact with an actual incident. If a frontier system produces the kind of behavior Charlton described in a real workplace first, the question of whether voluntary testing was the right tool becomes a question for an inquiry rather than a forum.