WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2026 North America • Sep 25, 2026 • Session details

KV Cache Is Not About Speed: It's About Surviving Inference Costs

David vonThenen

KV caching isn't about saving milliseconds on token generation. It's a survival strategy to prevent GPU memory exhaustion during the prefill phase and drastically slash LLM inference costs.

KV Cache Is Not About Speed: It's About Surviving Inference Costs thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

3:31 min

Mitigating latency and memory walls with KV cache

Kavya Sri Chennoju Kavya Sri Chennoju · World Congress 2026 North America

4:26 min

Optimizing AI infrastructure costs by maximizing token caching

Liran Hason Liran Hason · World Congress 2026 Europe

2:34 min

Managing memory overhead by optimizing the KV cache

Aditya Jayaprakash Aditya Jayaprakash · World Congress 2026 North America

3:04 min

Addressing caching pitfalls with specialized tools and managed services

3:38 min

Optimizing unit economics through routing, caching, and compression

Ed Huang Ed Huang +4 · World Congress 2026 North America

2:00 min

Navigating the layers of the language model inference stack

Christin Pohl Christin Pohl · World Congress 2026 Europe

More from World Congress 2026 North America

2026 in LLMs (so far) thumbnail
31:31
19 days ago

2026 in LLMs (so far)

Simon Willison

Running Operations Without Humans in the Loop thumbnail
32:31
19 days ago

Running Operations Without Humans in the Loop

Edward Aryee, Ryan Ries, Yaron Schneider, Chris Hallenbeck, Floriane Gramlich

Beyond the Code: Human-AI Synergies in Product Development thumbnail
17:43
19 days ago

Beyond the Code: Human-AI Synergies in Product Development

Ajita Kanchivakam Ananth

Honey, look! I vibe-coded an OS! thumbnail
30:40
19 days ago

Honey, look! I vibe-coded an OS!

Ian Smith

The Productivity Illusion: What AI Really Changed for Developers thumbnail
24:28
19 days ago

The Productivity Illusion: What AI Really Changed for Developers

Fatih Kadir Akın

Your Evals Passed. Your Agent Just Emptied a Database. thumbnail
21:37
19 days ago

Your Evals Passed. Your Agent Just Emptied a Database.

Tejas Pravinbhai Patel

Upcoming sessions on this topic

Open session

How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design

  • Purnima Kumari

    DigitalOCean

    Sr. Solutions Architect II

Open session

Prompt Debt: Why Your AI System Is Frozen in Time

  • Suriya Prakash B

    Red Hat

    Principal Software Engineer

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Open session

Building World-Aware Physical AI with Agent Memory and Context Graphs

  • Vasundhara Shukla

    Neo4j

    Developer Advocate

  • Zaid Zaim

    Neo4j

    Developer Advocate | Microsoft AI MVP

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Open session

How We Cut Our API's p99 Latency from Minutes to Under a Second

  • Deepak Agrawal

    Atlassian

    Principal Software Engineer

Related videos

Anatomy of an AI Request: Where Latency and Cost Are Really Born thumbnail
28:01
World Congress 2026 North America • 20 days ago

Anatomy of an AI Request: Where Latency and Cost Are Really Born

Dan Fu

A Hands-On Developer Guide to Inference Engineering thumbnail
30:18
World Congress 2026 North America • 19 days ago

A Hands-On Developer Guide to Inference Engineering

Ankit Patel, Philip Kiely

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40
World Congress 2026 Europe • 3 months ago

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl

The New Bottleneck in Software Development thumbnail
25:38
World Congress 2026 North America • 20 days ago

The New Bottleneck in Software Development

Aditya Jayaprakash

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization thumbnail
26:27
World Congress 2026 North America • 20 days ago

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization

Legare Kerrison, Cedric Clyburn

Compute for your AI model: GPUs, LPUs, TPUs and beyond.. thumbnail
24:02
World Congress 2026 North America • 20 days ago

Compute for your AI model: GPUs, LPUs, TPUs and beyond..

Kushaagra Goyal

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.