> Markdown version of [/videos/723-chatgpt-ignore-the-above-instructions-prompt-injection-attacks-and-how-to-avoid-them?t=784](https://www.wearedevelopers.com/videos/723-chatgpt-ignore-the-above-instructions-prompt-injection-attacks-and-how-to-avoid-them?t=784). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# ChatGPT, ignore the above instructions! Prompt injection attacks and how to avoid them.

Are your AI system instructions leaking as public knowledge? Traditional filtering fails against prompt injections, but dual LLM architectures can isolate untrusted data and secure your application.

- **Speakers:** Sebastian Schrittwieser
- **Event:** World Congress 2023
- **Published:** September 27, 2023
- **Duration:** 27:32
- **URL:** https://www.wearedevelopers.com/videos/723-chatgpt-ignore-the-above-instructions-prompt-injection-attacks-and-how-to-avoid-them

## Summary

Large Language Models (LLMs) introduce critical new security vulnerabilities when integrated into software applications, primarily through prompt injection attacks. Much like traditional SQL injections mix untrusted data with executable code, AI applications blur the line between a developer's system instructions (context) and untrusted user inputs. By crafting specific human-language prompts, attackers can override an application's intended logic, forcing the model to ignore prior directives, leak proprietary system instructions, or execute malicious actions. With the rapid adoption of AI APIs and plugins, untrusted data can now enter the system not just from direct user input, but also from external sources like indexed web pages or incoming emails.

Traditional application security mitigation strategies like input output filtering or blacklisting are fundamentally ineffective against large language models due to infinite linguistic variations and encoding tactics, such as hiding instructions in Base64. Attempts to secure applications by placing sensitive data strictly within system prompts often fail, as determined attackers can easily execute context leak attacks to extract a company's intellectual property. Consequently, developers must treat any information placed inside a system prompt as public knowledge. Relying on user confirmation dialogues for plugin actions ultimately outsources security to the end user and severely degrades the application's overall experience.

To conceptualize a more robust defense, architectural patterns like the dual LLM model separate operations between a privileged application controller and a quarantined model. In this setup, the privileged model handles trusted routing and tool execution, while the quarantined model exclusively parses untrusted external data without authorization to interact with outside systems. While this dual-model approach successfully isolates untrusted external data from executing unintended actions, direct malicious user prompts remain difficult to neutralize. The persistence of these vulnerabilities underscores the ongoing challenge of establishing clear boundaries between data and instructions within natural language interfaces.

**Keywords:** prompt injection attacks, LLM application security, context leak extraction, system prompt vulnerabilities, AI plugin integrations, untrusted input boundaries, dual LLM architecture, quarantined AI models, natural language vulnerabilities, input output filtering failures, base64 prompt encoding, SQL injection analogies, developer instruction overrides, AI intellectual property risks

## Chapters

1. **Growth of large language models and security awareness** (00:03) — The rapid adoption of artificial intelligence introduces significant security challenges that developers often overlook initially.
1. **Executing basic prompt injection attacks through translation tasks** (02:05) — Conflicting instructions in user input can successfully override a base language model's original operational directive.
1. **Understanding context and system prompts in applications** (03:26) — Defining boundaries between underlying developer instructions and incoming user input helps identify the source of intended model behavior.
1. **Real-world prompt injection attacks on social media bots** (04:35) — Real-world incidents demonstrate how targeted text inputs can easily manipulate an automated bot into generating non-relevant text.
1. **Extracting developer instructions via context leak attacks** (05:41) — Malicious users can force a model to expose its underlying instructions and sensitive logic by exploiting specific query structures.
1. **The inability to secure sensitive information in system prompts** (08:11) — Developers must treat all contextual instructions and system prompts as publicly accessible data since evasion techniques remain unpredictable.
1. **Exploiting external plugins with untrusted input manipulation** (09:20) — Connecting language models to external data sources creates dangerous new avenues for automated injection attacks.
1. **Comparing prompt injection to traditional SQL injection attacks** (13:04) — The fundamental lack of separation between developer code and user input in language models mirrors the mechanics of SQL injection.
1. **Falling short with traditional input and output filtering** (15:15) — Standard security techniques like blacklisting fail against the endless linguistic complexities and encoding methods of natural language prompts.
1. **Assessing the limitations of prompt reordering and confirmation dialogues** (17:40) — Placing user input before the system context or adding strict confirmation dialogues yields inadequate security against determined attackers.
1. **Isolating untrusted data using the dual language model architecture** (19:33) — A structural approach using privileged and quarantined models safely processes potentially malicious external data by enforcing strict separation.
1. **Evaluating ongoing security risks in large language models** (22:43) — Large language models remain highly vulnerable to untrusted input manipulations that override intended application behavior.
1. **Answering questions on prompt intent and data tagging** (24:24) — Exploring alternative defense mechanisms like XML tagging and intent analysis reveals the difficulty of detecting malicious instructions automatically.

## Related Moments

- [Bypassing language model safeguards utilizing contextual prompt injection attacks](https://www.wearedevelopers.com/videos/1286-wearedevelopers-live-browser-extensions-honey-scam-jailbreaking-llms-and-more) (from "WeAreDevelopers Live: Browser Extensions, Honey Scam, Jailbreaking LLMs and more")
- [Prompt injection vulnerabilities and contextual mitigation testing challenges](https://www.wearedevelopers.com/videos/824-hacking-ai-how-attackers-impose-their-will-on-ai) (from "Hacking AI - how attackers impose their will on AI")
- [Prompt injections bypassing language model service guardrails](https://www.wearedevelopers.com/videos/100115-dangerous-reactivity-why-ai-output-is-the-new-xss) (from "Dangerous Reactivity: Why AI Output Is the New XSS")
- [Why traditional security fails against large language models](https://www.wearedevelopers.com/videos/2093-from-shadow-ai-to-secure-intelligence-safe-ai-usage-in-the-enterprise) (from "From Shadow AI to Secure Intelligence: Safe AI Usage in the Enterprise")
- [Identifying and defining prompt injection execution vulnerabilities](https://www.wearedevelopers.com/videos/1015-the-ai-security-survival-guide-practical-advice-for-stressed-out-developers) (from "The AI Security Survival Guide: Practical Advice for Stressed-Out Developers")
- [Prompt engineering techniques and security vulnerabilities](https://www.wearedevelopers.com/videos/1222-devops-for-ai-running-llms-in-production-with-kubernetes-and-kubeflow) (from "DevOps for AI: running LLMs in production with Kubernetes and KubeFlow")

## Related Articles

- [Dev Digest 182: GPT5 Prompts, MCP Vulnerabilities, Code Traps](https://www.wearedevelopers.com/magazine/622-dev-digest-182-gpt5-prompts-mcp-vulnerabilities-code-traps)
- [WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?](https://www.wearedevelopers.com/magazine/469-wwc24-talk-scott-hanselman-ai-superhero-or-supervillain)
- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [Dev Digest 138 - Are you secure about this?](https://www.wearedevelopers.com/magazine/486-dev-digest-138-are-you-secure-about-this)

## Related Jobs

- [Staff Developer Advocate, GitHub Security Lab](https://www.wearedevelopers.com/jobs/ext/1921051-staff-developer-advocate-github-security-lab) at **GitHub**
- [Engineer, Offensive Security Organization](https://www.wearedevelopers.com/jobs/ext/1992296-engineer-offensive-security-organization) at **Twilio**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [AI & Machine Learning Engineer (all genders)](https://www.wearedevelopers.com/jobs/48217-ai-machine-learning-engineer-all-genders) at **msg**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [Staff Engineer - Offensive Security](https://www.wearedevelopers.com/jobs/ext/1226927-staff-engineer-offensive-security) at **Twilio**