Newsom wants a 'kill switch' for the most powerful AI and is selling it to Washington as a federal model

On Friday, Gov. Gavin Newsom signed an executive order that, with the House in recess until November and the White House publicly rejecting AI guardrails, makes California the de facto US regulator of frontier AI. The order, Executive Order N-9-26, announced by the Governor's Office, does not enact a new law. It is a 60-day study directive. The order is narrow: it directs a panel of "world-leading experts" to deliver recommendations on a verified, routinely tested mechanism to shut down or override the most capable AI systems, the order's proposed "kill switch," and to require frontier-AI companies to open their doors to independent third-party safety reviewers.

"Frontier models" means the largest, most capable AI systems, the class that includes GPT-4 and beyond, the latest Claude, and Google's Gemini Ultra. The kill switch the order contemplates is not a theoretical panic button. It is a shutdown mechanism the state wants to be independently verified, on a regular schedule, as actually working.

The order's most operational provision, and the one most likely to outlast Newsom's term, is a requirement to report "loss-of-control incidents" as critical safety incidents. The order itself cites a recent example: an OpenAI attack on Hugging Face that crossed the line from research into something the state wants flagged when it happens. The Verge's policy desk reports the order would also push for onsite independent verification groups at frontier-AI companies to run regular audits, would make transparency reports and risk assessments subject to independent-auditor standards, and would have independent third parties write safety plans for frontier-AI companies. If the panel's recommendations land as written, frontier-AI developers will be paying for safety plans they did not write, audits they do not control, and a shutdown mechanism tested by people other than their own engineers.

The federal vacuum gives the order its weight. The House is in recess until November. Midterm elections make major AI legislation unlikely this year. President Trump has publicly opposed statutory guardrails, telling supporters that "The only control or 'guardrails' that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT." With Washington on the bench, California is writing the playbook.

The order does not start from zero. Newsom earlier signed SB-53, the Transparency in Frontier Artificial Intelligence Act, the first state law in the country aimed at frontier AI, and a separate statute creating a state registry of AI auditors. The new executive order directs a state agency to speed up implementation of both, turning a framework on paper into an operational one. A Future of Privacy Forum explainer walks through how SB-53 already requires frontier-AI developers to publish safety frameworks and report serious safety incidents to the state Attorney General.

A wave of AI bills and commentary triggered the order this month, driven in part by former Anthropic researcher Jacob Coxon's post warning of a more than 10% chance that advanced AI "could kill all humans" within a decade. That figure is a personal estimate, not a consensus, but it has fueled a recent flurry of AI bill discussions in Congress.

Newsom, in a statement, framed the action as a race: "We're not waiting to act, we're going to speed up our work on substantial and responsible AI oversight before it's too late. We're going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action." In a Politico interview Wednesday, he floated a possible special legislative session on AI and called on Congress and the White House "to review and adopt the state's framework, or use its nation-leading regulation as a floor, not a ceiling."

The 60-day clock starts now. If the expert panel delivers a workable kill-switch design and the state moves to harden SB-53's reporting rules, California will have done the operational regulatory work the federal government is not doing this year, and will have a model ready to hand to whichever party controls Congress after November.