WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE β€’ Oct 30, 2024

Lies, Damned Lies and Large Language Models

Jodie Burchell

Why do large language models confidently lie? Naive RAG pipelines often make factuality hallucinations worse. Discover how to systematically evaluate foundation models and implement adaptive retrieval.

Pause
Mute Enter Fullscreen
#1 about 3 min

Introduction to large language model hallucinations

Autoregressive text generation models produce convincing but functionally incorrect outputs due to underlying dataset abstractions.

#2 about 2 min

Transformer architecture and training data scalability

Decoder-based algorithms scale continuously by using next-word prediction across massive datasets to improve sequence generation capabilities.

#3 about 3 min

Evolution of GPT models and lossy compression

Increasing parameters in successive model generations enables the retention of structural grammar alongside specific parametric knowledge.

#4 about 3 min

Filtering unstructured web data for model training

Filtering offensive or duplicate content from datasets like Common Crawl severely impacts generated model output quality.

#5 about 2 min

Distinguishing faithfulness and factuality hallucination errors

Models exhibit errors by either ignoring explicit prompt context or relying on incorrect training data.

#6 about 3 min

Measuring text hallucinations with standardized evaluation datasets

Identifying factual inaccuracies and ingrained misconceptions requires standardized benchmark frameworks like TruthfulQA and Squad.

#7 about 6 min

Demonstrating an open source TruthfulQA evaluation pipeline

Testing multiple-choice misconception queries against language models requires setting up benchmarks using LangChain and Hugging Face.

#8 about 2 min

Analyzing large language model benchmark evaluation results

Interpreting factuality error rates from testing frameworks enables leveraging community-driven leaderboards instead of manual validation.

#9 about 3 min

Techniques for mitigating model hallucination output frequencies

Mitigation strategies include context-heavy constraint prompting, domain-specific fine-tuning, and algorithmic output evaluation techniques used alongside retrieval augmentation.

#10 about 5 min

Structuring effective retrieval augmented generation system architectures

Integrating dynamic vector database bounds into text inference adds targeted context while introducing potential data retrieval inaccuracies.

#11 about 2 min

Implementing advanced context patterns to reduce hallucinations

Using conditional retrieval methodologies like ROWAN limits unnecessary vector queries when baseline model confidence is high.

Matching moments

3:32 min

Fundamentals and limitations of large language models

Krzystof Czieslak Β· LIVE

1:53 min

Overcoming language model challenges using retrieval-augmented generation

Ashish Sharma Β· LIVE

8:57 min

Analyzing hallucinations in generative AI models

Chris Heilmann Chris Heilmann +2 Β· LIVE

3:29 min

Mitigating artificial intelligence hallucinations with constraints and context

Jemiah Sius Jemiah Sius Β· World Congress 2024

1:26 min

Overcoming hallucination challenges in large language models

Michael Hunger Michael Hunger Β· World Congress 2024

1:49 min

Addressing the risks of hallucination and chat model manipulation

Zachary Powell Zachary Powell Β· World Congress 2024

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Read more, Prompt Less thumbnail
1:01:30
3 days ago

WeAreDevelopers LIVE - Read more, Prompt Less

Chris Heilmann, Daniel Cranney, Luise Freese

WeAreDevelopers LIVE - Tech Education in Two Languages thumbnail
1:00:18
10 days ago

WeAreDevelopers LIVE - Tech Education in Two Languages

Daniel Cranney, Roxy Rodriguez-Becker

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40
17 days ago

WeAreDevelopers LIVE - Playing in Production

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
about 1 month ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
about 1 month ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof CieΕ›lak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
about 1 month ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Open session

How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design

  • Purnima Kumari

    DigitalOCean

    Sr. Solutions Architect II

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Open session

Agentic Forecasting with sktime: Towards Autonomous Time Series Analysis

  • Franz Kiraly

    sktime

    Director

Open session

Prompt Debt: Why Your AI System Is Frozen in Time

  • Suriya Prakash B

    Red Hat

    Principal Software Engineer

Related videos

Truth, Lies, and Probabilities: Testing AI Hallucinations thumbnail
32:14
World Congress 2026 Europe β€’ 3 months ago

Truth, Lies, and Probabilities: Testing AI Hallucinations

Anastasia Simou

Large Language Models ❀️ Knowledge Graphs thumbnail
29:11
World Congress 2024 β€’ about 2 years ago

Large Language Models ❀️ Knowledge Graphs

Michael Hunger

Inside the Mind of an LLM thumbnail
27:11
World Congress 2025 β€’ about 1 year ago

Inside the Mind of an LLM

Emanuele Fabbiani

Building Blocks of RAG: From Understanding to Implementation thumbnail
26:25
LIVE β€’ almost 2 years ago

Building Blocks of RAG: From Understanding to Implementation

Ashish Sharma

Martin O'Hanlon - Make LLMs make sense with GraphRAG thumbnail
24:42
LIVE β€’ over 1 year ago

Martin O'Hanlon - Make LLMs make sense with GraphRAG

Martin O'hanlon

Give Your LLMs a Left Brain thumbnail
27:35
World Congress 2024 β€’ about 2 years ago

Give Your LLMs a Left Brain

Stephen Chin

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.