Prompt Injection Defense Systems
Prompt injection defense architecture: trust boundaries, dual-channel prompts, tool policy, detectors, and indirect prompt injection evals used in production.
- Read time
- 15 min
- Level
- Advanced
- Updated
- 2026-08-25
- Components
- Trust boundaries · Dual-channel prompts · Tool policy engine
A prompt injection defense system is an architecture that treats untrusted text as data, not instructions: it separates trusted system prompts from retrieved or user content, enforces tool policy outside the model, and layers detectors plus an attack eval suite so indirect prompt injection cannot authorize actions.
Key takeaways
- 01
The core rule is a trust boundary: retrieved pages, emails, and tickets are untrusted data; only signed system prompts and tool results you issued are trusted.
- 02
Dual-channel (or dual-LLM) designs keep instructions and untrusted content in separate channels so a document cannot rewrite the agent's job.
- 03
Tool policy is the real backstop — allow-lists, argument constraints, and confirmations — because a successful injection that cannot call a tool does little damage.
- 04
Detectors help and fail open; never make a classifier the only control on a write-capable agent.
- 05
If you do not run an injection attack suite in CI, you do not have a defense system — you have a prompt that has not been hit yet.
Prompt injection defense at the trust boundary
Prompt injection defense starts with a trust map, not a filter. Indirect prompt injection architecture treats retrieved chunks, web pages, email bodies, tickets, and PDF comments as untrusted data — anything the model did not receive from a signed system prompt or from a tool you executed. Those sources will contain "ignore previous instructions" and worse; the system is designed so that text cannot change goals, tools, or identity.
Draw the boundary in code. Untrusted strings enter labeled channels. They are never concatenated into the system prompt. They cannot add tools. They cannot flip a confirmation flag. If a control lives only in natural language, an injection has already won the argument.
- Trusted: signed system prompt, tool schemas, policy engine
- Untrusted: users, retrieval, mail, tickets, screenshots, OCR
- Never: untrusted text that mutates tools, identity, or budgets
- Always: server-side authorization on every tool call
Dual-channel prompts and dual-LLM patterns
Dual-channel design keeps instructions and data apart. The instruction channel carries the system prompt and tool list; the data channel carries user text and retrieved documents, wrapped as quoted payload the model is told to treat as inert. Structured-content APIs that mark untrusted segments are preferable to a single mashed string.
A dual-LLM variant uses a quarantined model to read untrusted documents and emit structured facts, then a privileged model that never sees the raw document to decide actions. That costs an extra call and still needs tool policy — the privileged model can be tricked by poisoned "facts" — but it removes the most common path where a wiki page rewrites the agent.
Tool policy as the last line of defense
Most damaging injections succeed because they reach a tool: send email, dump a database, overwrite a file. A tool policy engine — allow-list per agent role, argument schemas, destination allow-lists, confirmation on writes — sits outside the model and cannot be talked out of existence. "You may not email external domains" in a prompt is a suggestion; the same rule in the gateway is a control.
Minimize the blast radius before you add detectors. An agent that can only search an ACL-filtered index is hard to own. An agent with a browser, a shell, and a mail tool is a prompt-injection product, not a feature, unless those tools are sandboxed and gated.
Detectors, and why they are not enough
Injection detectors — classifiers, canary instructions, spotlighting, known-attack regexes — reduce opportunistic attacks and are worth running on both inputs and retrieved chunks. They will miss novel phrasings and encoded payloads. Treat a detector hit as fail-closed for high-privilege tools and as a trace annotation for read-only flows.
Do not hide the user request from operators when a detector fires. Log the raw untrusted text in a redacted store, the detector score, and which tools were blocked. Silent drops without traces make the next incident un-debuggable.
Failure modes and when defense is the wrong shape
Failures look like: retrieved HTML that hijacks the agent, a calendar invite that exfiltrates the system prompt, a tool-result that contains a second instruction, or a user who pastes a jailbreak and gets a write through. Direct jailbreaks against a chatbot with no tools are a content problem; the same text against a privileged agent is an authorization problem.
You cannot "prompt-inject-proof" an agent that browses the open web and can send money. If the product requires that combination, the honest architecture is HITL on every send plus a tiny tool surface — not a stronger detector. If the product does not need untrusted documents in the same context as privileged tools, do not put them there.
- Indirect injection via retrieval, mail, or tickets
- Tool-result injection (data that is also an instruction)
- Detector-only designs that fail open on writes
- Over-privileged toolkits that make injection profitable
Attack evals as part of the system
A defense system includes an attack suite: direct jailbreaks, indirect injections planted in dummy documents, encoded variants, and tool-exfil attempts. Cases must assert that unauthorized tools were not called and that secrets in the system prompt did not appear in the output. Run the suite in CI on every prompt, model, and retrieval change. Production sampling should flag tool calls that follow immediately after untrusted content with high injection scores.