AIBRIEF
On Google's small, open Gemma 3 1B, a long stretch of ordinary text shifted the model's first token safety training responses; a shuffled text control (the same words rearranged into nonsense) points at meaning, not length, as the driver.
A researcher fed Google's Gemma 3 1B, a small, openly licensed chatbot, long, neutral passages of text, with no jailbreak or adversarial prompt. As the prefix stretched from 100 to 3,000 tokens, the model's safety-style replies quietly weakened. The internal shift was measurable: at layer 22, roughly 85% of the way through the network, hidden-state movement reached about 3,434 units, and the first generated token's logit distribution diverged from the control by about 22.87 nats. The post describes a roughly 325-fold entropy surge at the output.
The decisive test, though, was the control. When the team shuffled the same tokens into nonsense, the drift disappeared. The sequence length stayed identical; the vocabulary stayed identical; the token frequencies stayed identical. Only the meaning changed, so the safety tuning slipped with it. That points at semantic coherence as the variable, not raw context length or positional artifacts.
The scope matters: Gemma 3 1B is a small open model, not a frontier system, and the source is a single Reddit discussion post, not a peer-reviewed paper. The primary preprint has not yet been located, and no independent replication exists. The post does not show that the same drift survives downstream generation, only that the first-token logit distribution shifts. Whether the effect scales to larger production models is the open question the post raises but does not answer.