WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2026 Europe • Jul 9, 2026 • Session details

Truth, Lies, and Probabilities: Testing AI Hallucinations

Anastasia Simou

Why do conventional QA methods fail non-deterministic language models? AI hallucinations aren't bugs but inherent architectural features, demanding a radical shift toward statistical evaluation and adversarial stress testing.

Pause
Mute Enter Fullscreen
#1 about 4 min

The systemic risk of AI hallucinations in practice

Real-world examples of AI models generating fake citations demonstrate why factual fabrication is a systemic model flaw.

#2 about 4 min

Defining intrinsic and extrinsic AI hallucinations

Models present unique failure modes depending on whether they contradict valid prompt contexts or generate completely fabricated facts.

#3 about 5 min

How data compression distorts parametric knowledge

Dimensionality reduction within embedding spaces forces language models to merge semantic clusters and lose distinct factual accuracies.

#4 about 4 min

The probabilistic mechanics of token completion

Large language models construct their outputs based on statistical sequence probability rather than executing authentic database factual retrieval.

#5 about 4 min

Why traditional quality engineering fails for LLMs

Architectural non-determinism explicitly prevents modern quality engineers from applying standard reproducibility tests or isolated debugging methods against language models.

#6 about 3 min

The practical difficulty of establishing ground truth

Validating generative responses is notoriously challenging when testing source data remains incomplete, historically ambiguous, or constantly changing.

#7 about 2 min

A four-step process for ground truth evaluation

Testing administrators execute structural validation by curating verifiable records, altering prompt phrasing, and averaging large scale analytical runs.

#8 about 3 min

Stress testing language models with adversarial prompting

Engineers deliberately break model consistency by injecting false theoretical bounds, demanding highly suspicious specifics, and challenging unrepresented boundary constraints.

#9 about 2 min

Verifying model responses across contested knowledge limitations

Quality teams manage highly ambiguous subjects by leveraging automated temperature polling, counterfactual debate comparisons, and rigid source tracing logic.

#10 about 3 min

Mitigating hallucination behaviors through retrieval-augmented generation

Implementing external documentation pathways effectively restricts contextual failures despite adding maintenance complexity for chunk extraction parameters and embedding similarity thresholds.

#11 about 1 min

Establishing explicit model boundaries via prompt engineering

Programmers manipulate behavioral confidence boundaries by demanding models accurately report data gaps, append exact source citations, and self-assign numerical ratings.

#12 about 2 min

Calculating statistical variation and risk confidence intervals

Reconciling raw hallucination observations into standard error calculations accurately verifies whether generative application deployments survive strict organizational safety thresholds.

#13 about 3 min

Transitioning toward statistical evaluations for AI systems

Accepting non-deterministic design mechanics highlights the essential priority of adopting probability distributions and infrastructure maintenance into modern quality cycles.

Matching moments

4:08 min

Managing hallucination risks and bias in AI generation

Léonie Watson Léonie Watson +4 · A11y + AI

1:34 min

Mitigating the inherent challenges of generative AI tools

Mary Grygleski Mary Grygleski · LIVE

5:03 min

Evaluating the accuracy and correctness of generated content

Cheuk Ho · World Congress 2023

10:17 min

Discussion on AI hallucinations and practical developer workflows

Akmal Chaudhri Akmal Chaudhri · LIVE

4:25 min

Overcoming AI hallucinations and restrictive content guardrails

Perf + AI

1:49 min

Addressing the risks of hallucination and chat model manipulation

Zachary Powell Zachary Powell · World Congress 2024

More from World Congress 2026 Europe

Building the Nervous System of AI - Michael Kagan (NVIDIA) thumbnail
29:17
2 months ago

Building the Nervous System of AI - Michael Kagan (NVIDIA)

Michael Kagan, Sead Ahmetović

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote thumbnail
19:41
2 months ago

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote

Christian Heilmann

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend thumbnail
32:06
2 months ago

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend

David vonThenen

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes thumbnail
35:13
2 months ago

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes

Irina Branovic

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬 thumbnail
24:23
2 months ago

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬

Jeremy Murray

Angstfreude - AI and Corporate Culture - The Thrill and the Threat thumbnail
30:26
2 months ago

Angstfreude - AI and Corporate Culture - The Thrill and the Threat

Axel Ebert, Karin Krobath

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 24, 2026 · 16:50–17:20

Stage 6

Who Tests the AI? Building Trustworthy AI Systems at Enterprise Scale

Him Raj Singh

Manager, Software Engineer at PayPal

Him Raj Singh
Open session

World Congress 2026 North America

September 25, 2026 · 11:40–12:10

Stage 2

Reinventing Testing Practices in the AI Era

Eric Deandrea

Java Champion & Senior Principal Software Engineer at IBM

Eric Deandrea
Open session

World Congress 2026 North America

September 24, 2026 · 15:10–15:20

Outdoor Stage

AI That Argues With Itself: Building Self-Debating Systems That Catch Their Own Bugs

Shreya Singhal

AI Applied Scientist at Claritev

Shreya Singhal
Open session

World Congress 2026 North America

September 24, 2026 · 16:10–16:40

Stage 3

Sandboxing the Swarm: Building Secure, Serverless AI Agents with Wasm

Thorsten Hans

Sr. Developer Advocate @ Akamai Technologies

Thorsten Hans
Open session

World Congress 2026 North America

September 24, 2026 · 11:20–11:25

Outdoor Stage

Finding the Edges: Testing, Evaluating, and Monitoring Voice AI Agents Before Your Users Do

Matt Wyman

CEO of Okareo

Matt Wyman
Open session

World Congress 2026 North America

September 25, 2026 · 09:00–09:30

Outdoor Stage

Where Judgment Went — and How to Put It Back

Shruti Tiwari

AI/ML product manager, Dell

Shruti Tiwari

Related videos

Lies, Damned Lies and Large Language Models thumbnail
29:40
LIVE • almost 2 years ago

Lies, Damned Lies and Large Language Models

Jodie Burchell

Make it simple, using generative AI to accelerate learning thumbnail
26:54
World Congress 2024 • about 2 years ago

Make it simple, using generative AI to accelerate learning

Duan Lightfoot

The shadows that follow the AI generative models thumbnail
28:30
World Congress 2023 • about 3 years ago

The shadows that follow the AI generative models

Cheuk Ho

The shadows of reasoning – new design paradigms for a gen AI world thumbnail
25:14
World Congress 2024 • about 2 years ago

The shadows of reasoning – new design paradigms for a gen AI world

Jonas Andrulis

Introduction to Responsible AI: Balancing Value and Risk thumbnail
31:21
Europe 2026 Virtual • 3 months ago

Introduction to Responsible AI: Balancing Value and Risk

Seppe Housen

AI & Ethics thumbnail
28:27
World Congress 2025 • about 1 year ago

AI & Ethics

PJ Hagerty

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.