WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2026 North America • Sep 24, 2026 • Session details

Anatomy of an AI Request: Where Latency and Cost Are Really Born

Dan Fu

Running pre-fill and decode on the same GPU severely bottlenecks your AI throughput. Discover how cache-aware disaggregation and speculative decoding can drastically reduce latency and infrastructure costs.

Starting at 12:14 · Watch from the beginning

Anatomy of an AI Request: Where Latency and Cost Are Really Born thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

5:58 min

Architecting inference layers and meta harnesses to manage token costs

Hari Lingamagunta Hari Lingamagunta +3 · World Congress 2026 North America

1:47 min

Managing compute costs and AI model routing

Deivids Vilkinsons Deivids Vilkinsons +3 · World Congress 2026 Europe

3:20 min

Optimizing AI infrastructure from applications to GPU kernels

Erran Berger Erran Berger +1 · World Congress 2026 North America

1:52 min

Prioritizing developer user experience over raw model parameters

Neel Sundaresan Neel Sundaresan +1 · World Congress 2026 Europe

2:37 min

Understanding core parameters and mechanics of large language models

Julián Duque Julián Duque · World Congress 2025

1:47 min

Balancing scale and architecture in artificial intelligence development

Yuval Dvir Yuval Dvir · World Congress 2026 North America

More from World Congress 2026 North America

2026 in LLMs (so far) thumbnail
31:31

2026 in LLMs (so far)

Simon Willison · Datasette /Django

Running Operations Without Humans in the Loop thumbnail
32:31

Running Operations Without Humans in the Loop

Edward Aryee +4

Beyond the Code: Human-AI Synergies in Product Development thumbnail
17:43

Beyond the Code: Human-AI Synergies in Product Development

Ajita Kanchivakam Ananth · Google

Honey, look! I vibe-coded an OS! thumbnail
30:40

Honey, look! I vibe-coded an OS!

Ian Smith · LYOS

The Productivity Illusion: What AI Really Changed for Developers thumbnail
24:28

The Productivity Illusion: What AI Really Changed for Developers

Fatih Kadir Akın · WordPress/Automattic

Your Evals Passed. Your Agent Just Emptied a Database. thumbnail
21:37

Your Evals Passed. Your Agent Just Emptied a Database.

Tejas Pravinbhai Patel · Amazon

Upcoming sessions on this topic

Open session

How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design

  • Purnima Kumari

    DigitalOCean

    Sr. Solutions Architect II

Open session

Bypassing the GPU Tax: Smart Layer-7 Routing with Small Language Models (SLMs)

  • Sagar Utekar

    CrowdStrike

    SRE

Open session

Engineering AI Strategy: Beyond the Hype

  • Omkar Srivastava

    ADSK

    Engineering Leader

Open session

Your AI Agents Are Bleeding Tokens: Building a Runtime Cost Control Plane for LLMs

  • Aakriti Aggarwal

    IBM Research

    Research Engineer

  • Bhavishya Pandit

    66degrees

    Leading AI Innovation

Open session

Enterprise AI Memory at Scale

  • Aditya Joshi

    Walmart

    Senior Software Engineer

Open session

Building a Semantic Cache for Agentic AI: Practical workshop on reducing cost & latency

  • Bhavana Giri

    Redis

    Developer Advocate

Related videos

Compute for your AI model: GPUs, LPUs, TPUs and beyond.. thumbnail
24:02

Compute for your AI model: GPUs, LPUs, TPUs and beyond..

Kushaagra Goyal · Rubrik

World Congress North America · 2026

A Hands-On Developer Guide to Inference Engineering thumbnail
30:18

A Hands-On Developer Guide to Inference Engineering

Ankit Patel +1

World Congress North America · 2026

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl · Microsoft

World Congress Europe · 2026

The Unit Economics of AI thumbnail
29:56

The Unit Economics of AI

Ed Huang +4

World Congress North America · 2026

KV Cache Is Not About Speed: It's About Surviving Inference Costs thumbnail
25:52

KV Cache Is Not About Speed: It's About Surviving Inference Costs

David vonThenen · NetApp

World Congress North America · 2026

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

World Congress · 2024

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.