WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2024 • Aug 20, 2024 • Session details

Manipulating The Machine: Prompt Injections And Counter Measures

Georg

Hackers are tricking AI chatbots into selling cars for $1 and stealing 2FA tokens. Learn why complex system prompts fail and how to properly defend against prompt injections.

Pause
Mute Enter Fullscreen
#1 about 4 min

Anatomy of large language model prompts and structural layers

The three distinct layers of user input, context, and system instructions shape the fundamental behavior of generative language models.

#2 about 2 min

Understanding the risks of unchecked user inputs in chatbots

Unfiltered prompts leave applications vulnerable to unintended commands and logic overrides remarkably similar to traditional database injection attacks.

#3 about 2 min

Mitigating basic prompt manipulation attempts using explicit system rules

Adding concrete semantic limits and behavioral boundaries in the system prompt block filters out naive and rudimentary adversarial commands.

#4 about 4 min

Bypassing system constraints by extracting hidden instructions and rules

Attackers leverage conversational logic conventions to force applications into improperly revealing internal instruction sets directly back to the user.

#5 about 2 min

Extracting confidential business information from configured knowledge bases

Manipulated user inputs can subvert logic flows to silently expose proprietary backend files originally intended solely for context enhancement.

#6 about 2 min

Manipulating automated talent evaluation systems with hidden text injections

Embedding invisible instructions into resumes deliberately forces automated screening pipelines to render favorable assessments without exhaustive human oversight.

#7 about 3 min

Exploiting integrated application programming interfaces for unauthorized data extraction

Deliberately crafted external messages reliably hijack autonomous integration processes to systematically intercept authentication tokens sent to an inbox.

#8 about 4 min

Establishing foundational limitations to reduce system prompt injection vulnerabilities

Restricting integrations exclusively to read-only capabilities drastically minimizes potential exploitation vectors by neutralizing external write permissions.

#9 about 2 min

Defending against targeted prompt attacks by deploying fine-tuned models

Training isolated learning environments directly on specialized domain data drastically diminishes inherent susceptibility to adversarial instructional overrides.

#10 about 3 min

Filtering malicious application inputs utilizing adversarial prompt detection classifiers

Secondary language model deployments running pattern recognition databases act as robust firewalls by classifying problematic submissions prior to evaluation.

#11 about 3 min

Acknowledging current limitations in securing generative artificial intelligence applications

Since zero operational guarantees currently exist avoiding application compromises, administrators should practically consider all system text publicly viewable information.

Matching moments

2:52 min

Mitigating indirect prompt injection in language models

Jose Manuel Ortega Jose Manuel Ortega · Europe 2026 Virtual

5:44 min

Prompt injection vulnerabilities and contextual mitigation testing challenges

Mirko Ross · World Congress 2023

2:46 min

Bypassing language model safeguards utilizing contextual prompt injection attacks

Chris Heilmann Chris Heilmann +1 · LIVE

8:04 min

Understanding AI chatbot vulnerabilities and stateful attacks

Sebastian Messingfeld Sebastian Messingfeld · World Congress 2026 Europe

3:04 min

Prompt injections bypassing language model service guardrails

Ramona Schwering Ramona Schwering · World Congress 2026 Europe

1:07 min

Prompt injection and artificial intelligence agent behavior

More from World Congress 2024

Beyond the Hype: Real-World AI Strategies Panel thumbnail
24:07
about 1 year ago

Beyond the Hype: Real-World AI Strategies Panel

Mike Butcher, Jürgen Müller, Katrin Lehmann, Tobias Regenfuss

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation thumbnail
21:17
almost 2 years ago

Carl Lapierre - Exploring Advanced Patterns in Retrieval-Augmented Generation

Carl Lapierre

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI! thumbnail
31:50
about 2 years ago

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Zaid Zaim

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging thumbnail
29:43
about 2 years ago

Building a diverse tech company: The power of Diversity, Equity, Inclusion, and Belonging

Sue Quackenbush

Livecoding with AI thumbnail
31:39
about 2 years ago

Livecoding with AI

Rainer Stropek

Dapr in Practice thumbnail
24:08
about 2 years ago

Dapr in Practice

marcklefter

Upcoming sessions on this topic

Open session

Prompt Debt: Why Your AI System Is Frozen in Time

  • Suriya Prakash B

    Red Hat

    Principal Software Engineer

Open session

From Prompt Engineering to Harness Engineering: Building Observable and Governed GenAI Systems

  • Lakshmi Devi Prakash

    JPMorgan Chase

    VP, Applied AI/ML Lead

Open session

Beware of Strangers Bearing Code: Open Source Trust in the Agent Era

  • Vikram Vaswani

    Consultant

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Open session

Autopsy of an Autonomous Incident: When the Agent Made It Worse

  • Navin Pai

    StackGen

    Director of Engineering

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Related videos

ChatGPT, ignore the above instructions! Prompt injection attacks and how to avoid them. thumbnail
27:32
World Congress 2023 • about 3 years ago

ChatGPT, ignore the above instructions! Prompt injection attacks and how to avoid them.

Sebastian Schrittwieser

The Things Your AI Isn't Telling You thumbnail
25:06
World Congress 2026 North America • 18 days ago

The Things Your AI Isn't Telling You

Desmond Lamptey

Prompt Injection, Poisoning & More: The Dark Side of LLMs thumbnail
23:24
World Congress 2025 • about 1 year ago

Prompt Injection, Poisoning & More: The Dark Side of LLMs

Keno Dreßel

The AI Security Survival Guide: Practical Advice for Stressed-Out Developers thumbnail
30:36
World Congress 2024 • about 2 years ago

The AI Security Survival Guide: Practical Advice for Stressed-Out Developers

Mackenzie

Hacking AI - how attackers impose their will on AI thumbnail
27:02
World Congress 2023 • almost 3 years ago

Hacking AI - how attackers impose their will on AI

Mirko Ross

Hack Me If You Can: Designing Unbreakable LLM Guardrails thumbnail
30:53
World Congress 2026 Europe • 3 months ago

Hack Me If You Can: Designing Unbreakable LLM Guardrails

Cansu Kavili Örnek

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.