Back to Blogs
Industry Insights

Prompt Injection: Can It Ever Be Fully Fixed? What AI Labs and Security Agencies Admit

OWASP, the UK NCSC and OpenAI explain why prompt injection remains a difficult AI security challenge and how layered defense reduce real-world risk.

S

Simranjit Kaur

AI Trainer

Oct 27, 2026
13 min read
Prompt Injection AI Defense

Prompt injection is one of the most important security challenges facing modern AI systems. As large language models (LLMs) move beyond answering questions and begin reading emails, searching websites, processing documents and operating tools, a simple question becomes increasingly important: what happens when the information an AI reads contains instructions designed to manipulate it?

A common belief is that prompt injection can eventually be solved by adding a smarter model, improving the system prompt or placing a filter in front of the application. These measures can help, but they do not guarantee that every attack will be prevented. The deeper issue is that language models process instructions and external content within the same general language-processing context, making it difficult to establish a perfectly reliable security boundary.

In guidance published in 2025, the Open Worldwide Application Security Project (OWASP), the UK's National Cyber Security Centre (NCSC) and OpenAI highlighted different aspects of this problem. Their recommendations point towards a more realistic strategy: reduce the probability of successful attacks, limit what a compromised AI can do, and detect suspicious behaviour before it causes serious damage.

This article explains the problem, what these organizations actually say, and what developers should implement when building applications that use AI agents or LLMs.

What Is Prompt Injection?

Prompt injection occurs when content supplied to an AI system influences its behaviour in an unintended way. The content may attempt to override the system's intended instructions, redirect the task, expose sensitive information or persuade an AI agent to misuse a connected tool.

Consider an AI assistant that has been asked to summarize an email. The email appears ordinary, but it also contains a hidden or misleading instruction telling the assistant to ignore the user's request and disclose private information. If the assistant treats that untrusted content as an instruction rather than merely something to analyze, the attack may succeed.

The risk becomes more serious when an AI assistant can access email accounts, internal documents, web browsers, databases or external services. A manipulated answer is one problem; a manipulated answer that triggers a consequential action is another.

Direct vs. indirect prompt injection

  • Direct prompt injection: an attacker places malicious instructions directly in a prompt or message supplied to the AI.

  • Indirect prompt injection: the AI encounters malicious instructions inside external content, such as a webpage, document, email, repository or other material it was asked to process.

Indirect attacks are particularly important for AI agents because the user may never see the malicious instructions. The AI can encounter them while performing a completely legitimate task.

Why a Smarter Model or Better Filter Is Not a Complete Fix

Input filters, system prompts, safety training and improved models are useful defenses. However, none should be treated as a guarantee that prompt injection will never work.

Attackers can change the wording, hide instructions within otherwise useful content, split instructions across different sections or exploit content from sources that the application considers relevant. In multimodal systems, malicious instructions may also be embedded in images or other inputs.

The underlying difficulty is the relationship between instructions and data. A traditional database can enforce a structural boundary between a query and the values supplied to it. Parameterized SQL queries, for example, help ensure that user-supplied values are treated as data rather than executable SQL instructions.

Language models work differently. They process tokens and generate responses based on their context. Although models can be trained to prioritize trusted instructions and recognize untrusted content, this does not create an infallible technical boundary between the two.

That distinction changes the security strategy. Instead of assuming that a model will always recognize and reject a malicious instruction, developers should design systems that remain protected even if the model is influenced by one.

What OWASP Says: Prompt Injection Is LLM01:2025

The OWASP GenAI Security Project lists LLM01:2025 Prompt Injection as the first risk in its 2025 Top 10 for Large Language Model Applications. Its guidance describes how prompts or external content can alter an LLM's behaviour in unintended ways.

OWASP also explains that techniques such as retrieval-augmented generation (RAG) and fine-tuning do not fully mitigate prompt injection. These techniques can improve retrieval, relevance or model behaviour, but they should not be mistaken for complete security controls.

Its mitigation guidance recommends multiple measures, including:

  • Constrain the model's behaviour and intended scope.

  • Validate expected output formats using deterministic application code.

  • Apply appropriate input and output filtering.

  • Enforce least-privilege access for tools and connected systems.

  • Require human approval for high-risk operations.

  • Separate and identify untrusted external content.

  • Conduct regular adversarial testing and attack simulations.

The practical lesson is that prompt injection must be addressed across the application, not only inside the prompt sent to the model.

What the UK NCSC Admits: Risk Reduction, Not a Silver Bullet

On 10 December 2025, the UK's National Cyber Security Centre published a warning titled Mistaking AI vulnerability could lead to large-scale breaches, highlighting the dangers of treating prompt injection like a conventional software injection vulnerability.

In its associated technical blog, Prompt injection is not SQL injection (it may be worse), the NCSC explains that current LLMs do not enforce a dependable security boundary between instructions and data inside a prompt. It argues that prompt injection may never be totally mitigated in the same way as SQL injection.

This is a statement about the limitations of current LLM architectures and the difficulty of guaranteeing prevention. It does not mean every attack will succeed or that security engineering is pointless.

The NCSC's advice is to design AI systems around the possibility that a model may be manipulated. Developers should limit the tools and privileges available to the model, understand what an attacker could achieve, and monitor the application for suspicious activity.

The agency also warns against trusting products that claim to stop prompt injection outright. Organizations should instead examine how a solution reduces risk and what protections remain if a model behaves unexpectedly.

What OpenAI Says About Prompt Injection in AI Agents

OpenAI's published guidance on hardening ChatGPT Atlas against prompt injection describes the issue as an ongoing challenge for agent security. It says the company expects to keep working on the problem for years and considers prompt injection unlikely to ever be fully "solved".

This matters because browser agents can encounter untrusted instructions across webpages, emails, shared documents, calendar invitations and other sources. When an agent can take actions through a browser, a successful manipulation could potentially lead to unintended information sharing or changes to user data.

OpenAI describes a layered approach that includes model training, automated attack discovery, adversarial testing, system-level safeguards and monitoring. Its broader user guidance also highlights controls such as limiting access to necessary data and reviewing consequential actions before confirming them.

The important distinction is that OpenAI's position does not amount to saying that defenses cannot improve. It acknowledges that attacks evolve and that meaningful risk reduction requires ongoing improvements to both the model and the surrounding system.

Defense in Depth: How to Build Safer AI Systems

If no single filter can guarantee prevention, what should a developer do? The answer is defense in depth: combine controls so that one failure does not automatically become a security incident.

Defense in Depth Safer AI Systems

1. Apply least privilege

An AI agent should have access only to the information and tools required for its task. A document summarization assistant does not automatically need permission to send emails, delete files or make purchases.

Keep sensitive credentials outside the model's direct control. Enforce permissions in application code and the underlying services rather than relying on the model to decide whether an action is allowed. Where possible, use narrowly scoped credentials and read-only access.

2. Require confirmation for consequential actions

Actions that send messages, transfer money, publish content, modify records or share sensitive information should have appropriate approval gates. Before execution, present the user with the intended action and relevant details.

Confirmation should be meaningful rather than automatic. The user must be able to understand what will happen, review the destination and data involved, and cancel the operation. For especially sensitive actions, additional authorization may be necessary.

3. Monitor inputs, outputs and tool use

Monitoring helps teams detect unusual behaviour and investigate incidents. Depending on the application and its privacy requirements, useful security events may include suspicious instructions, unexpected tool calls, denied permissions, repeated failed actions and unusual data-access patterns.

Logs should be protected, access-controlled and retained according to a defined policy. Avoid collecting unnecessary sensitive content. Monitoring is useful only when someone can investigate alerts, respond to incidents and improve the controls that failed.

4. Separate untrusted content and enforce boundaries in code

Clearly label external documents, retrieved passages and user-supplied content as untrusted. Tell the model how this content should be used, but do not depend on those instructions alone.

Use deterministic checks to validate tool arguments, output schemas, resource identifiers and access permissions. A model-generated request should pass application-level authorization checks before it reaches a sensitive tool or service.

5. Test continuously, including indirect attacks

Security testing should cover direct prompts and malicious content encountered through webpages, files, emails, retrieval systems and other connected sources. Include attempts to expose data, bypass permissions and trigger unintended actions.

Test after model changes, new integrations and permission updates. Red-team exercises and automated evaluations can reveal weaknesses, but passing a test suite does not prove that every possible attack has been eliminated.

6. Reduce unnecessary autonomy

Do not give an AI agent broad freedom simply because the model appears capable. Break complex workflows into smaller steps, restrict the actions available at each step and require approval when risk increases.

The safest architecture is one in which a manipulated model cannot easily turn an untrusted instruction into a high-impact action.

Prompt Injection Defense: A Practical Comparison

Control

What it helps with

What it cannot guarantee

Input filtering

Detects known or suspicious patterns

Recognition of every novel or disguised attack

Stronger models and training

Improves resistance to malicious instructions

Perfect behaviour across every context

Content separation

Makes trusted instructions and external data clearer

An infallible boundary inside the model

Least privilege

Limits the resources an agent can access

Every form of harmful output or misuse within permitted access

Human confirmation

Creates an approval checkpoint for important actions

All errors, especially if users approve without checking

Monitoring and testing

Improves detection, investigation and future resilience

Discovery of every attack before an incident occurs

These controls work best together. Filtering may reduce the number of suspicious inputs reaching a model; restricted permissions limit the potential impact of a successful manipulation; confirmation gates protect consequential actions; and monitoring helps teams investigate and respond.

What This Means for AI Engineers and Security Professionals

Prompt injection is not just a topic for cybersecurity specialists. It affects AI engineers building RAG applications, developers integrating language models into business software, automation engineers connecting agents to tools, and security teams responsible for protecting data.

Building a useful AI system now requires more than writing effective prompts. Professionals need to understand data flows, authentication, authorization, secure API design, model limitations, logging, testing and human oversight.

For students and aspiring professionals, a practical learning path could include:

  • Step 1 : Learn the foundations: Python, APIs, HTTP, authentication and basic application security.

  • Step 2 : Build an LLM application: experiment with a chatbot, document question-answering system or RAG application using non-sensitive sample data.

  • Step 3 : Add security controls: restrict tools, validate inputs and outputs, implement permissions, and require confirmation for sensitive actions.

  • Step 4 : Test the system: use safe, controlled prompt-injection test cases and document which safeguards work and where the system remains vulnerable.

  • Step 5 : Demonstrate your work: publish a clear project report describing the architecture, threat model, tests, limitations and improvements.

Students looking for AI courses in Jalandhar should evaluate programmes on more than their tool lists. Look for practical projects, responsible AI practices, security fundamentals, instructor guidance and opportunities to demonstrate what you have built.

At Thread Security Education, the broader goal is to connect AI learning with practical implementation and security-aware engineering. Whether you are exploring AI development, LLM security or AI agent security, focus on building systems that are useful, testable and designed with realistic limitations in mind.

Explore Thread Security for information about its technology and training initiatives. Check the official website for current course details, availability and programme information.

Frequently Asked Questions

Can prompt injection ever be completely eliminated?

There is no established method that guarantees complete prevention across all LLM applications and attack scenarios. OWASP, the UK NCSC and OpenAI describe prompt injection as an ongoing challenge. Defenses can improve substantially, but systems should be designed to manage residual risk.

Can a prompt-injection filter solve the problem?

A filter can detect known patterns and reduce exposure, but attackers can change or disguise instructions. Filters should be one layer of defense, supported by least privilege, application-level permission checks, human approval and monitoring.

Does RAG prevent prompt injection?

No. Retrieval-augmented generation can help an application retrieve relevant information, but retrieved documents may contain malicious instructions. Treat retrieved content as untrusted and enforce security controls outside the model.

Why is prompt injection especially important for AI agents?

Agents may use tools, access private information and perform actions on a user's behalf. If an agent follows malicious instructions from an external source, the consequences can extend beyond an incorrect answer to unauthorized actions or data exposure.

What is the most important defense against prompt injection?

There is no universal single control. A strong starting point is to limit what the AI can access and do, enforce permissions in code, require confirmation for high-impact actions, and continuously test and monitor the application.

The Takeaway: Design for Risk, Not Perfect Immunity

The idea that a smarter model or a better filter will eventually make prompt injection disappear is not a safe assumption for system designers. OWASP's LLM01:2025 guidance, the UK's NCSC analysis and OpenAI's published work all point to the need for layered, continuously improving defenses.

The practical objective is not to trust an AI agent to recognize every malicious instruction. It is to ensure that even when the agent is manipulated, its permissions, approval gates and monitoring reduce the chance of serious harm.

For developers, organizations and aspiring AI professionals, that is the essential lesson of modern LLM security: build useful systems, test them against realistic attacks, limit their authority and plan for the possibility that individual safeguards will fail.

Thread Security Education encourages learners to approach AI with both technical curiosity and security awareness. The future belongs not only to people who can build AI applications, but also to people who understand how to make those applications more resilient.

ARTICLE TOPICS
#Thread Security
#AI Courses in Jalandhar
#AI Agent Security
#Prompt Injection
#AI Safety
#Cybersecurity
#OWASP LLM01
#LLM Security
#Generative AI Security
#Prompt Injection Attacks
CUTTING-EDGE AI TRACK
100% Practical Defense Labs

AI Security & LLM Red Teaming Masterclass

Learn to identify and mitigate prompt injections, jailbreaks, data poisoning, and model theft in production AI systems with sandboxed exercises.

S

Simranjit Kaur

VERIFIED

AI Trainer Ā· Thread Security

Academic Verified Ā· E-E-A-T

Faculty mentor & research guide at Thread Security Education.

Specializations: AI Trainer
CONNECT:LinkedIn

Related Research & Articles

View All
Business Email Compromise Invoice Fraud
Industry Insights

Business Email Compromise: How Invoice Fraud Works and How Finance Teams Can Stop It

Business Email Compromise can turn routine invoices into costly fraud. Learn how payment redirection works and how finance teams can stop it.

By Akshay Kumar

Ransomware Payment  Risk vs Recovery
Industry Insights

Should Companies Pay Ransomware Demands? What Law Enforcement, Insurers and Data Say

Ransomware payment decisions are complex. Explore what law enforcement, cyber insurers and recovery evidence say before a company pays.

By Akshay Kumar

AI Banking Fraud Detection
Industry Insights

AI Fraud in Banking: How Banks Flag Suspicious Transactions

AI fraud detection in banking helps flag suspicious transactions, but false positives and human review remain important.

By Simranjit Kaur