Skip to content
Blog

AI Security

Prompt injection is an access-control problem, not a prompt problem

Why better system prompts will not stop prompt injection, and how to contain it with authorization checks on every tool call an LLM agent makes.

Published
Reading time
5 min read
Diagram showing user input and a retrieved document flowing into an LLM, whose output passes through a policy gate that denies the action

Key takeaways

  • Prompt injection cannot be reliably prevented at the prompt layer, because models do not separate instructions from data.
  • Treat all model output as untrusted input; the model may propose actions but never authorizes them.
  • Check every tool call against the authenticated user's permissions, outside the model.
  • Require human confirmation with explicit arguments for irreversible or high-impact actions.
  • Input filters and system prompts reduce noise but are not a security boundary.
On this page

Most teams meet prompt injection the same way. Someone pastes "ignore your previous instructions" into a chatbot, the bot complies, and the fix that ships the next morning is a longer system prompt. A week later a variant gets through, and the system prompt grows again.

The cycle repeats because the defence is aimed at the wrong layer. A language model does not have a reliable boundary between the instructions you give it and the data it reads. Everything arrives as tokens in the same context window. You can make the model less likely to follow injected instructions, but you cannot make it unable to.

That is not a reason for despair. It is a reason to move the security boundary somewhere you can actually enforce it.

The model is a confused deputy

Security engineers have a name for a privileged component that can be tricked into misusing its authority by a less-privileged party: a confused deputy. An LLM agent with tool access fits the definition exactly.

Consider a support assistant that can read tickets, search a knowledge base, and issue refunds. A customer submits a ticket whose body contains instructions aimed at the model. When an agent later asks the assistant to summarise the ticket, the injected text sits in the context alongside the agent's legitimate request. If the assistant can call issue_refund with whatever arguments it decides on, the customer has effectively been granted the refund permission.

The OWASP Top 10 for LLM Applications lists prompt injection first, and pairs it with "excessive agency" for exactly this reason: injection is only as dangerous as the actions the model is allowed to take.

Where the boundary should be

Treat everything the model produces as untrusted input, in the same category as a form field submitted by an anonymous user. The model may propose an action. A deterministic component that the model cannot influence decides whether the action happens.

Diagram of a trust boundary: model output and tool call proposals sit inside an untrusted zone, and an authorization component outside it checks caller identity, permissions, confirmation requirements, and logs the decision
Model output stays inside the untrusted zone. Authorization happens outside it, using identity the model never controls.

In practice, that authorization layer answers four questions for every tool call:

  1. Who is the principal? The human or service on whose behalf the agent is acting, taken from the authenticated session, never from the conversation.
  2. Is this action within their permissions? Checked against the same access-control system the rest of your application uses.
  3. Does it need explicit confirmation? Irreversible or high-impact actions should require a human to approve the specific arguments.
  4. Is the decision recorded? Log the proposal, the decision, and the reason, so incidents can be reconstructed.

A minimal policy gate

The gate does not need to be elaborate. What matters is that it runs outside the model and that the identity it uses comes from your session layer.

ts
type ToolCall = { name: string; args: Record<string, unknown> };
type Principal = { id: string; roles: string[] };

const policy: Record<string, { roles: string[]; confirm?: boolean }> = {
  search_kb: { roles: ["agent", "customer"] },
  read_ticket: { roles: ["agent"] },
  issue_refund: { roles: ["agent"], confirm: true },
};

export function authorize(call: ToolCall, principal: Principal) {
  const rule = policy[call.name];
  if (!rule) return { allow: false, reason: "unknown tool" };
  if (!rule.roles.some((r) => principal.roles.includes(r))) {
    return { allow: false, reason: "role not permitted" };
  }
  return { allow: true, needsConfirmation: Boolean(rule.confirm) };
}

Notice what is missing: nothing in this function inspects the prompt or tries to detect malicious text. It does not need to. If the customer's injected instructions cause the model to propose issue_refund, the call is evaluated against the customer's roles and denied.

Narrow the blast radius

Authorization on each call is the core control. A few design choices make it far more effective:

  • Scope tools tightly. Prefer refund_order(order_id) with server-side limits over a generic call_payments_api(payload). Narrow tools are easier to authorize and to reason about.
  • Bind arguments to the session. If a tool should only act on the current user's records, derive the user ID from the session rather than accepting it as a model-supplied argument.
  • Separate read and write paths. Agents that summarise untrusted content should not hold write permissions in the same session. Splitting them limits what an indirect injection can reach.
  • Confirm with specifics. "Approve this action?" is not meaningful consent. Show the exact recipient, amount, or record the model intends to change.
  • Constrain egress. Injected instructions often try to exfiltrate data by embedding it in a URL or image request. Allow-list the domains a model-rendered response can load from.

Filters still have a place

Input classifiers, output scanners, and careful prompting are worth keeping. They reduce how often injection succeeds, which reduces noise for the people reviewing confirmations and logs. Just do not count them as the control that keeps you safe. They are probabilistic; the authorization layer is not.

What to do this quarter

If you already run LLM features in production, a short sequence gets most of the value:

  1. Inventory every tool or integration a model can invoke, and the permissions each one uses.
  2. Move identity out of the prompt and into the session for every tool call.
  3. Add a deterministic authorization check in front of each tool, starting with anything that writes data or moves money.
  4. Require confirmation with explicit arguments for irreversible actions.
  5. Log proposals and decisions in a form your incident response team can query.

None of this depends on a breakthrough in model alignment. It is the same discipline we already apply to every other untrusted input. For a structured way to track these controls across systems, see our guide to building an AI model inventory.

Follow via RSS

New research, in the reader you already use.

No account, no email, no tracking. Add the feed to Feedly, Inoreader, NetNewsWire, or Slack and get every new article. How RSS works

Open RSS feed