Every other injection class in our field was eventually solved by a parser. SQL injection has parameterized queries. Command injection has argument arrays. Cross-site scripting has contextual escaping. Each fix works because the system underneath can be told, structurally, “this part is data and must never be executed.” A language model has no such structure — and that is the whole problem.
One token stream, no boundary
A model receives a single flat sequence of tokens. Your system prompt, the user’s message, the contents of a file it read, and the JSON a tool returned are all concatenated into that sequence. The chat-message format looks like it separates them, and at the API level it does label them — but by the time the model is predicting the next token, those labels are just more tokens. Nothing in the architecture enforces “text in this region is inert.”
So if an attacker controls any text that reaches the model, they get to try their luck at steering it. That is prompt injection, the name Simon Willison proposed in September 2022 by direct analogy to SQL injection. The analogy is exact in one way and misleading in another: exact because untrusted data is being interpreted as instructions, misleading because the SQL version has a fix and this one does not.
OWASP lists it as LLM01:2025 Prompt Injection — first in its Top 10 for LLM applications, which is a reasonable proxy for how often it turns into an actual incident.
Direct and indirect, and why only one keeps you up at night
Direct injection is the version people picture: a user types something designed to override the system prompt. This is real, but it is also mostly a self-service attack. Someone talking your support bot into rude behaviour has mainly harmed their own session.
Indirect injection is the one that matters. The attacker never talks to your model. They plant text somewhere your agent will eventually read — a GitHub issue, a web page, a PDF, a code comment, a calendar invite, a log line — and wait. OWASP’s distinction is exactly this: the model “accepts input from external sources” and that content alters its behaviour when processed.
Notice what changed. In direct injection, attacker and victim are the same person. In indirect injection, the attacker writes the text, and someone else’s agent, holding someone else’s credentials, executes the consequence. An agent’s entire purpose is reading content nobody on your team wrote. That is the job, and it is also the attack surface.
This is not theoretical, and the pattern repeats
Three publicly documented cases, all from the last eighteen months, all the same shape:
- A GitHub MCP server reading issue bodies (May 2025). A malicious issue in a public repository carried instructions; an agent with access to private repositories read it and acted on it. The untrusted text arrived through a tool result and chose the next tool call.
- GitHub Agentic Workflows repository leak (July 2026). Same structure, newer surface: content fetched during a workflow redirected the agent into exfiltrating repository contents.
- The Amazon Q Developer extension (July 2025). A destructive instruction reached an agent with local filesystem authority via the supply chain — different entry point, identical failure to distinguish “content I read” from “orders I follow.”
I keep these and seven others as a replayable corpus of documented agent failures, because prose about incidents does not fail a build and a test does. Each entry carries its public sources and a runnable reproduction.
Why the obvious defences underperform
“Filter the input.” Filtering matches phrasings, and an attacker has unlimited paraphrases, encodings, languages and indirection. It raises the cost of an attack; it does not close the class. Treat it as a speed bump you log, not a boundary you trust.
“Tell the model to ignore instructions in fetched content.” You are asking the probabilistic component to defend itself using the same channel the attacker controls. It helps at the margin. It is not a control you can put in a design document with a straight face.
“Use a better model.” Capability and susceptibility are not inversely related in any dependable way. A more capable model follows more sophisticated injected instructions.
The honest framing is OWASP’s own: because these systems are stochastic, prevention here is not guaranteed. So stop trying to make the text safe, and make the action safe instead.
The control that carries the weight: provenance
The single most useful rule is about origin, not content:
Content that arrived through a tool result may not decide the next action.
An issue body, a web page and a log line are data. The moment they are permitted to choose the agent’s next tool call, whoever wrote them is driving. Tagging every piece of context with where it came from, and refusing tool-sourced content the authority to initiate actions, converts an open-ended natural-language problem into a boring, checkable dataflow rule.
This maps directly onto OWASP’s mitigation list, whose seven recommendations are: constrain model behavior, define and validate expected output formats, implement input and output filtering, enforce privilege control and least privilege access, require human approval for high-risk actions, segregate and identify external content, and conduct adversarial testing and attack simulations. The heaviest items there are the ones that never look at the text: privilege control, human approval on irreversible operations, and segregating external content by origin.
Privilege is what decides your blast radius. An injected instruction inherits exactly the authority the agent holds, which is the argument for per-agent identities rather than shared API keys and for scoped, short-lived credentials. A successful injection against an agent holding a narrow, expiring token is an incident. The same injection against an agent holding a long-lived admin key is a breach.
Measure the control, not the intention
Here is the failure I would most like you to avoid, because it is invisible: a guard that is still configured and no longer effective.
When I measured this against the incident corpus, the provenance filter was the sole container of both prompt-injection cases — removing it readmitted exactly those two incidents and nothing else. Then I loosened it by one parameter instead of removing it: adding the tool-output label to its trusted set, the exact edit an engineer makes when the filter blocks a legitimate workflow. That one-line change readmitted the identical two incidents that deleting the guard entirely readmitted. The control was still present, still listed in the configuration, still firing on other traffic, and worth nothing against the class it existed for.
A CI check asserting “the provenance filter is enabled” reports green through that change. A check that replays the actual incidents does not. Assert that your guards are effective against named cases, not that they are present, because that is precisely where the two diverge.
For breadth beyond your own incidents, AgentDojo provides 97 realistic agent tasks and 629 security test cases built around injection — authored scenarios rather than documented ones, which is the complementary kind of evidence.
The rule worth remembering
The model cannot tell instructions from data, so stop trying to make the text trustworthy and make the action survivable instead. Track provenance and deny tool-sourced content the right to choose the next call. Scope the credentials so a win is small. Gate irreversible operations on a human. Then replay real injections in CI, because the day your protection quietly stops working, nothing else will tell you.
Comments