Context Poisoning: When an Agent's Own Hallucination Becomes Its Own Ground Truth

Context poisoning is the failure mode where a single hallucinated claim or erroneous observation makes it into an agent's own context, whether that is a goals list, a running summary, or a scratchpad the agent writes to itself, and is then repeatedly read back and treated as settled fact for the remainder of the session. Nothing external needs to reinforce the claim. The agent's own architecture does the work: whatever gets written into a persisted goal or summary carries the same authority as anything else in that section, regardless of whether a tool confirmed it or the model simply made it up on a bad turn. Once poisoned, the agent does not need to hallucinate again; it just needs to keep reading what it already wrote.

The clearest documented case comes from Google DeepMind's own long-context agent testing. In Pokémon Red and Blue, a thirsty guard is passed a drink bought from a vending machine to let the player through; in the Fire Red and Leaf Green remakes, the same puzzle instead requires a TEA item that does not exist in the original games. Gemini 2.5 Pro, playing the original version, at several points became convinced it needed to find and deliver TEA anyway, and spent extended stretches of play searching for an item that was not in the game it was actually playing (Breunig, "An Agentic Case Study: Playing Pokémon with Gemini," dbreunig.com, 2025). Once that belief reached the agent's persisted goals list, the behavior it produced was not a one-off wrong move: the model became, in language quoted directly from Google's account of the incident, "fixated on achieving impossible or irrelevant goals" (Google DeepMind, Gemini 2.5 Technical Report, arXiv:2507.06261; case study by Breunig, 2025). The false premise did not need to be repeated by anything outside the agent. It sat in the agent's own memory and kept paying out.

This is a different mechanism from three adjacent patterns already covered here. Memory poisoning requires an external write path into a persistent, cross-session store, typically an adversary or an untrusted document supplying the false content; context poisoning needs neither an attacker nor persistence beyond the current session, since the agent contaminates its own single-session context by hallucinating and then reading its own hallucination back. Hallucination cascades are a multi-agent phenomenon: a fabrication crosses from one agent to a different downstream agent that has no way to tell it apart from a verified claim. Context poisoning is single-agent: the same agent originates the false claim and later re-consumes it, with no handoff required at all. And it differs from context rot, which is accuracy degrading as a general function of how much content sits in a long window; context poisoning is not a volume effect, it is one specific false claim getting written down and then carrying full evidentiary weight going forward, something that can happen even in a short context if a single compaction pass writes the wrong thing into a goal.

Mechanism

The Pokémon case study is useful precisely because the failure is mechanical rather than mysterious. A long-running agent typically keeps three kinds of state in play: the live turn-by-turn transcript, a periodically regenerated summary that compresses that transcript to stay under a token budget, and a goals or plan section that the summary and transcript both feed into. Nothing in this arrangement tags a given line with where it came from. A goal derived from a tool's confirmed output and a goal derived from the model's own unconfirmed guess look identical once both are sitting in the same list, and both get carried forward into the next planning step with equal weight. Google's own account of the agent's behavior noted a related, compounding effect at long horizons: as the context grew significantly beyond 100,000 tokens, the agent showed a tendency toward favoring repeating actions from its own history rather than synthesizing new plans (Google DeepMind, Gemini 2.5 Technical Report, arXiv:2507.06261). A model already leaning on its accumulated history as the default source of what to do next is exactly the condition under which a single bad entry in that history is most likely to get repeated rather than questioned.

Compaction accelerates the effect rather than containing it. Each time a summary is regenerated, the noisy, contradictory detail that might have undercut a false claim is exactly the kind of material a summarization pass is designed to discard, while the claim itself, if it was ever phrased as a goal or a settled fact, is the kind of material a summary is designed to keep. The poisoned entry survives compaction better than the evidence that could have caught it, and each successive summary reads a little more confidently than the last, because the record has been quietly edited down to agree with itself.

This sits inside a broader four-part taxonomy of context failure modes, named and organized by the same source as the Pokémon case study: alongside context poisoning, "when a hallucination or other error makes it into the context, where it is repeatedly referenced," the taxonomy names context distraction, "when a context grows so long that the model over-focuses on the context, neglecting what it learned during training," context confusion, "when superfluous content in the context is used by the model to generate a low-quality response," and context clash, "when you accrue new information and tools in your context that conflicts with other information in the context" (Breunig, "How Long Contexts Fail," dbreunig.com, 2025). Distraction is the mechanism this site already treats as context rot; clash is the mechanism this site already treats as instruction fragmentation. Poisoning is the one of the four without a dedicated entry here, and the distinguishing feature worth holding onto is that it is not a volume or accumulation problem like its three siblings, it is a provenance problem: the failure is that nothing in the pipeline distinguishes a claim the agent verified from a claim the agent invented, once both are written down in the same place.

How to detect it

Diff an agent's goals or plan section against its own tool-call and observation log at several points in a long session. A goal with no corresponding confirmed observation behind it is a candidate for a poisoned entry, whether or not it has caused visible harm yet.
Watch for a plan that narrows toward one specific, oddly rigid sub-objective that the rest of the session's confirmed facts do not support, especially one introduced by a summary rather than by a tool result. The Pokémon case shows the signature clearly: a single invented detail become the dominant driver of subsequent actions.
Track whether repeated failures to achieve a stated goal produce more attempts at the same approach rather than a reconsideration of the goal itself. An agent that keeps trying harder at an impossible goal, rather than questioning whether the goal is real, is behaving exactly as the poisoned Pokémon agent did.
Re-run the same session transcript with the suspected false entry manually removed from the summary and compare the next several actions. A sharp behavioral change confirms the entry, not something else, was driving the pattern.

Mitigation

Tag provenance on anything promoted into a goal, plan, or long-lived summary: whether a line originated from a tool's confirmed output or from the model's own unconfirmed inference. This does not require a new architecture, only that the compaction step preserve one bit of metadata it currently throws away, and it directly targets the root mechanism, since the failure is specifically that verified and self-asserted content become indistinguishable once written down.

Treat anything an agent must not get wrong the same way this site's state externalization pattern treats anything an agent must not forget: keep it in an authoritative, externally-writable record the agent reads from rather than in a self-summarized goals list the agent both writes and reads. A goal sourced from an external ledger cannot be silently invented by the model in the first place, which removes the entry point context poisoning depends on.

Use re-anchoring triggered specifically after any compaction event, not just on a fixed cadence, to re-derive the current goal from the original task statement and the confirmed observation log rather than from the most recent summary. Since poisoning propagates precisely by surviving into successive summaries, a re-anchoring step that bypasses the summary chain and goes back to source material breaks the propagation path directly.

Where an agent is expected to run for a long horizon, budget for a periodic independent review rather than assuming the agent will self-correct. The Pokémon agent's own hallucinations were not self-limiting; a false goal, once written, persisted until something outside the normal planning loop caught it.

Related terms

Memory poisoning: a persistent, cross-session store corrupted by external content, typically adversarial; context poisoning is single-session and self-generated, with no attacker or persistent store required.
Hallucination cascades: a fabrication crossing between different agents in a pipeline; context poisoning is one agent re-consuming its own fabrication with no handoff involved.
Context rot: accuracy degrading as a function of accumulated context volume; context poisoning is a single false claim carrying full weight regardless of how much context surrounds it.
State externalization: the durable, externally-sourced record pattern that removes the self-asserted goal entry this failure mode depends on.
Re-anchoring patterns: deliberately restoring a goal from source material, the direct countermeasure to a poisoned entry surviving inside a summary chain.

FAQ

Is this the same thing as memory poisoning? No. Memory poisoning requires an external write path into a store that survives across sessions, usually with an adversary or untrusted content supplying the false material. Context poisoning needs neither: the agent hallucinates the false claim itself, inside a single session, and no attacker or persistence beyond that session is required for the failure to compound.

Does this only happen in long sessions? It is more likely in long sessions because compaction, which tends to entrench a poisoned entry, happens more often the longer a session runs. But the underlying mechanism, a claim with no provenance tag getting written into a goal or summary and then read back as fact, can occur after a single compaction pass in an otherwise short session.

Can a bigger context window fix it? No. A larger window does not add provenance information to what is already written down; if anything, Google's own account of the Pokémon agent found that behavior degraded, not improved, once the context grew well beyond 100,000 tokens, with the agent favoring repetition of its own history over fresh reasoning.

How is this different from an agent just being wrong once? A single wrong turn that gets corrected on the next turn is an ordinary model error. Context poisoning specifically requires the error to be written into a part of the agent's state that persists and gets re-read, a goal or a summary, so that later turns build on it as if it were confirmed, which is what turns one bad guess into a sustained, self-reinforcing pattern of behavior.