Anatomy of an AI Request: Where Latency and Cost Are Really Born
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Running pre-fill and decode on the same GPU severely bottlenecks your AI throughput. Discover how cache-aware disaggregation and speculative decoding can drastically reduce latency and infrastructure costs.
Checking access…
Playback and chapters load privately for Free videos.
Matching moments
More from World Congress 2026 North America
Upcoming sessions on this topic
Starts in 5 days
Conference India 2026
November 25–26, 2026
Bengaluru, India
-
How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design
Purnima Kumari · DigitalOCean
Thu 26 Nov · Stage 2
-
Bypassing the GPU Tax: Smart Layer-7 Routing with Small Language Models (SLMs)
Sagar Utekar · CrowdStrike
Thu 26 Nov · Stage 2
Related videos
Related articles
DC
Daniel Cranney
BB
Benedikt Bischof
BB
Benedikt Bischof
DC
Daniel Cranney
From learning to earning
Jobs that call for the skills explored in this talk.
5 days ago
Lead Engineer, Google Cloud
PwC
Atlanta, GA, United States
Expert
$99k–232k
LangChain
Google Cloud
Generative AI
7 days ago
Senior AI Scientist - Model Training & Architecture
BWI GmbH
Hamburg, Germany
Expert
€87k–113k
NoSQL
QLoRA
Docker
9 days ago
AI Scientist - Model Training & Architecture
BWI GmbH
Bamberg, Germany
Expert
€87k–113k
NoSQL
QLoRA
Docker
7 days ago
Lead Engineer, Google Cloud
PwC
Boca Raton, FL, United States
Expert
$99k–232k
LangChain
Google Cloud
Generative AI
5 days ago
Lead Engineer, Google Cloud
PwC
Toledo, OH, United States
Expert
$99k–232k
LangChain
Google Cloud
Generative AI
about 2 months ago
•
AI Software Engineer (Germany)
Sunhat
Berlin, Germany
Expert
€70k–100k
Remote
REST
TypeScript
AI Frameworks