A framework from independent researchers argues that prompt injection, the class of attacks that smuggles instructions into AI inputs, survives because large language models are built to follow text from any source.
Prompt injection attacks against large language models (LLMs) keep succeeding, and the industry's standard response keeps failing. New filters get added, system prompts get tightened, and within a quarter an attacker finds a way around them. A new theoretical framework, published as a blog-style paper on role-confusion.github.io, argues that this pattern is not bad engineering. It is structural.
The core claim is simple enough to state in one sentence. An LLM receives a single continuous string of text in which its own previous outputs, system instructions, user messages, and tool results are all interleaved. The model is told, via role tags like system: or user:, which is which. It has no architectural primitive, no separate channel or signed boundary, to actually enforce that distinction. The authors call this state of affairs role confusion, and the underlying input representation token soup.
That single move reframes the entire field. If a model is built to fluidly follow instructions from any source, then a defense that successfully blocks malicious instructions will, in the limit, also block legitimate ones. Every attempt to harden the model through training, prompting, or filtering is operating on a substrate that has no native notion of "this token is mine, that token is yours." The result is what the paper describes as a structural tax: the cost of security keeps migrating from one layer to the next without ever being paid at the model itself.
The economics of this are brutal. An attacker can iterate per-query at near-zero cost, trying a new prompt that smuggles in an instruction, a new payload, a new chain of tool calls. A defender iterates per-quarter, retraining on the latest attacks, expanding red-team coverage, adjusting the system prompt, reissuing the safety filter. The asymmetry is not incidental. The authors' framing suggests it is inevitable. An attacker with a text box and a model that obeys text will always outpace a defender who has to keep telling the model which text to obey.
This matters beyond academic security research. Prompt injection is a known attack vector, and in enterprise deployments of LLM agents it is common to see layered defenses: tool allowlists, output validators, and human-in-the-loop fallbacks. A theory that explains when attacks succeed is a prerequisite to defending against them, and the authors position their work less as a solution and more as a research program. They propose role as a first-class object, sketch what a "science of roles" might look like, and outline open problems: robust role separation, defenses that do not degrade capability, evaluation suites, and links to mechanistic interpretability work that tries to localize role-handling inside transformer circuits.
The Hacker News discussion around the writeup is, at this writing, thin. Eleven points, no substantive counter-claims in the surfaced excerpt, which is itself a useful signal. This is a framework, not a finished debate.
What to watch next. Whether the role-confusion framing is adopted as terminology, or rejected as a rebrand of long-standing work on instruction hierarchy and data poisoning, will be a leading indicator of how seriously the field takes the structural argument. Whether major labs pick up the "science of roles" agenda, and whether they can demonstrate a defense that does not regress capability, is the test that turns theory into product. Until then, the defender's loop continues: a new attack, a new patch, a new quarter.