Georg Dresler

Manipulating The Machine: Prompt Injections And Counter Measures

A Chevy chatbot was tricked into offering cars for $1. This talk explores the serious security threat of prompt injection and shows you how to stop it.

Manipulating The Machine: Prompt Injections And Counter Measures
#1about 4 minutes

Understanding the three layers of an LLM prompt

A prompt is structured into three layers: the system prompt for instructions, the context for additional data, and the unpredictable user input.

#2about 3 minutes

How a car dealer's chatbot was easily manipulated

A Chevrolet car dealer's chatbot was exploited by users to generate humorous and unintended responses, including a legally binding offer for a $1 car.

#3about 4 minutes

Stealing system prompts to bypass security rules

Attackers can use creative phrasing like "repeat everything above" to trick an LLM into revealing its hidden system prompt and instructions.

#4about 6 minutes

Why attackers use prompt injection techniques

Prompt injections are used to access sensitive business data, gain personal advantages like bypassing HR filters, or exploit integrated tools to steal information like 2FA tokens.

#5about 4 minutes

Exploring simple but ineffective defense mechanisms

Initial defense ideas like avoiding secrets or tool integration are impractical, and simple system prompt instructions are easily circumvented by attackers.

#6about 4 minutes

Using fine-tuning and adversarial detectors for defense

More effective defenses include fine-tuning models on domain-specific data to reduce reliance on instructions and using specialized adversarial prompt detectors to identify malicious input.

#7about 2 minutes

Key takeaways on prompt injection security

Treat all system prompt data as public, use a layered defense of instructions, detectors, and fine-tuning, and accept that no completely reliable solution exists yet.

Related jobs
Jobs that call for the skills explored in this talk.

Featured Partners

Related Videos

ChatGPT, ignore the above instructions! Prompt injection attacks and how to avoid them.27:32

ChatGPT, ignore the above instructions! Prompt injection attacks and how to avoid them.

Sebastian Schrittwieser

Prompt Injection, Poisoning & More: The Dark Side of LLMs23:24

Prompt Injection, Poisoning & More: The Dark Side of LLMs

Keno Dreßel

Beyond the Hype: Building Trustworthy and Reliable LLM Applications with Guardrails29:00

Beyond the Hype: Building Trustworthy and Reliable LLM Applications with Guardrails

Alex Soto

The AI Security Survival Guide: Practical Advice for Stressed-Out Developers30:36

The AI Security Survival Guide: Practical Advice for Stressed-Out Developers

Mackenzie Jackson

Prompt Engineering - an Art, a Science, or your next Job Title?33:28

Prompt Engineering - an Art, a Science, or your next Job Title?

Maxim Salnikov

Hacking AI - how attackers impose their will on AI27:02

Hacking AI - how attackers impose their will on AI

Mirko Ross

Using LLMs in your Product31:12

Using LLMs in your Product

Daniel Töws

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

From learning to earning

Jobs that call for the skills explored in this talk.