AI text watermarks can weaken safety guardrails, researchers find

AI text watermarks are shifting how language models respond to adversarial prompts, according to research from Lasso Security, reported by Ars Technica. Anthropic has said its future Claude models will use the same family of watermark, putting the finding on a near-term shipping path.

Text provenance moved from research to compliance when the EU AI Act treated machine-readable provenance on AI-generated text as a control platforms must implement. Anthropic disclosed earlier this year that it will adopt Google's open-source SynthID-Text for upcoming Claude releases. Lasso's experiment is the first adversarial-prompt data on a watermark in that family.

SynthID-Text-style schemes embed a secret key into the model's next-word selection. At each step, instead of picking the single most likely token, the system runs a tournament of candidate words, scores them under the secret key, and steers the model toward options that will later be statistically detectable as machine-written. That bias is invisible to a reader, but it changes the probability distribution the model draws from, the same distribution a refusal-trained safety filter is calibrated against.

"Any change to the generation creates tradeoffs that show up somewhere," said Andrea Siposova, the researcher at Lasso Security who led the work. "The real question is whether the watermarked model is being tested under adversarial conditions before it ships."

A prompt a non-watermarked model would refuse, a request for sensitive personal data or a jailbreak that asks the model to role-play as a system administrator, can land on a different probability landscape in the watermarked version, and the safety filter is now grading a different slice of the output space. Lasso found that this can show up as more tool invocations, more compliance with the request, and a measurable drop in refusals on borderline prompts.

Any provenance control that nudges the model's generation, including a watermark, a metadata tag, or a downstream detector that feeds back into training, will reshape the input the safety filter operates on. The question for deployers is how to test for the shift before the model goes live.

For teams shipping watermarked models, the practical checklist looks like this: red-team under the watermark, not just on the base model; monitor refusal-rate drift between watermarked and non-watermarked builds; treat provenance as a security control with its own threat model, sitting alongside the safety stack rather than beneath it; and run the existing jailbreak suite with the watermark key on, because the failure modes will not be the same as on the unwatermarked model.

The EU rule is policy context, not the cause. The same regulatory pressure that pulled watermarks into production is now pulling adversarial testing into production. Lasso's result is one early data point, on a research watermark rather than a deployed Claude. Anthropic's first adversarial test under a shipped SynthID-Text build will be the next real measurement.