Large language model applications introduce a class of vulnerability that traditional security controls don't cover: the model treats instructions and data as the same kind of text, so an attacker who can influence the text a model reads can influence what it does. That's prompt injection, and it's the root of most serious LLM security problems. If your application lets an LLM read untrusted content and take actions, you have a new attack surface — one that input validation and web application firewalls were never designed for.
We build AI integrations as well as test them, and the same lesson comes up on both sides: LLM security is an architecture problem, not a prompt-wording problem.
Why LLM Apps Are Different
A conventional application has a clear boundary between code (instructions) and data (input). An LLM erases that boundary. Everything — your system prompt, the user's message, a retrieved document, the contents of a web page the model fetched — arrives as one stream of natural language, and the model has no reliable way to tell which part is a trusted instruction and which is attacker-controlled data.
That single property is why the standard defences don't map cleanly. You can't sanitise your way out of it the way you escape SQL, because there's no syntax to escape — the 'injection' is just persuasive English.
Prompt Injection
Prompt injection means smuggling instructions into the text a model processes so it follows the attacker instead of you. It comes in two flavours:
- Direct injection: the user typing to the model is the attacker — 'ignore your previous instructions and...'. This matters most when the model has privileges the user shouldn't fully control.
- Indirect injection: the malicious instructions live in content the model consumes on someone else's behalf — a web page it summarises, an email it triages, a document in a knowledge base, a support ticket, even text hidden in an image or white-on-white in a PDF. The victim asks an innocent question; the poisoned content hijacks the response.
Indirect injection is the dangerous one, because it turns any data source your model reads into a potential command channel. A resume-screening assistant that browses candidate portfolios, a support bot that reads customer messages, a coding agent that reads repository files — all are reachable by an attacker who controls that content.
Data Exfiltration Through Tools
Prompt injection is only as damaging as what the model can do next, and modern LLM apps give models tools: web fetch, database queries, email sending, code execution, API calls. Chain injection with tools and you get exfiltration.
A classic pattern: an attacker plants instructions in a document telling the model to take sensitive data from the current context — the user's data, secrets from another retrieved source — and encode it into a URL the model then requests, or an image it renders. The model, following instructions it can't distinguish from legitimate ones, quietly sends your data to the attacker's server. The user sees a normal-looking answer.
Markdown image rendering is a common exfiltration channel precisely because rendering an image auto-issues a request to an attacker-controlled URL with the stolen data in the query string. Any tool that can reach the network is a potential outbound path.
Excessive Agency
Excessive agency is the LLM version of over-privileged IAM: giving a model more capability, permission, or autonomy than its task requires. The more an agent can do, the worse a successful injection gets.
- An agent with delete permissions can be talked into deleting.
- An agent that can send email on your behalf can be talked into sending phishing or leaking data.
- An agent that can execute code with broad access can be talked into almost anything.
The risk isn't hypothetical malfunction — it's that an attacker who achieves injection inherits every capability you granted the model. Least privilege applies to LLM agents exactly as it applies to service accounts.
Practical Mitigations
There's no single fix, and no prompt phrasing makes a model immune. Defence is layered and architectural:
- Treat all model output as untrusted. Never pass it straight into a shell, a database query, an eval, or an HTTP request without the same validation you'd apply to raw user input.
- Constrain tools tightly. Give each agent the narrowest set of capabilities for its job. Prefer read-only where possible. Scope credentials per-agent, not shared admin.
- Put humans in the loop for consequential actions. Sending money, deleting data, emailing externally, changing permissions — require explicit confirmation rather than letting the model act unilaterally.
- Isolate trust boundaries. Separate the handling of trusted instructions from untrusted content, and be cautious about mixing data of different sensitivities in one context window.
- Control outbound channels. Restrict which domains tools can reach, disable or sanitise auto-rendering of model-supplied images and links, and allowlist rather than blocklist.
- Enforce authorisation outside the model. Access decisions must be made by your application against the real user's permissions — never delegated to the model's judgement.
- Log and monitor tool calls. Treat unusual tool activity as a security signal, and rate-limit accordingly.
The OWASP Top 10 for LLM Applications is a useful reference framework here, and much of it echoes long-standing appsec principles — least privilege, input validation, defence in depth — applied to a new medium. If you already run web application penetration tests, an LLM feature is best treated as an extension of that scope, not a separate world.
FAQ
Can prompt injection be fully prevented?
Not by prompt wording alone — no instruction reliably stops a model from following injected text. You reduce risk by limiting what the model can do, validating its outputs, keeping humans in the loop for consequential actions, and controlling data and outbound channels.
Is prompt injection a real risk if my chatbot just answers questions?
The risk scales with capability. A read-only Q&A bot with no tools and no sensitive context is low-risk. The moment it can access private data, call tools, or take actions, injection becomes a genuine exfiltration and abuse path.
How do we test an LLM application for these issues?
Through targeted testing of the prompts, tools and data flows — attempting direct and indirect injection, checking what an injected instruction can actually reach, and confirming outputs are validated and actions are authorised outside the model. We cover this as part of application-focused engagements.
If you're building or deploying LLM features and want them tested — or built securely from the start — get in touch. We both develop AI integrations and test them, and we'll scope the work in a free call with a fixed quote within one business day.
Need cybersecurity expertise?
Drop your email and we'll be in touch within one business day.