> Markdown version of [/videos/100526-anatomy-of-an-ai-request-where-latency-and-cost-are-really-born?t=734](https://www.wearedevelopers.com/videos/100526-anatomy-of-an-ai-request-where-latency-and-cost-are-really-born?t=734). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Anatomy of an AI Request: Where Latency and Cost Are Really Born

Running pre-fill and decode on the same GPU severely bottlenecks your AI throughput. Discover how cache-aware disaggregation and speculative decoding can drastically reduce latency and infrastructure costs.

- **Speakers:** [Dan Fu](https://www.wearedevelopers.com/@dan-fu)
- **Event:** World Congress 2026 North America
- **Published:** September 24, 2026
- **Duration:** 28:01
- **URL:** https://www.wearedevelopers.com/videos/100526-anatomy-of-an-ai-request-where-latency-and-cost-are-really-born
- **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/29a154d8857fb2efacec5d79b205a297/thumbnails/thumbnail.jpg?time=142949ms

## Access

Playback and chapters for this video are available with a Free account.

## Related Moments

- [Architecting inference layers and meta harnesses to manage token costs](https://www.wearedevelopers.com/videos/100600-inside-the-ai-native-engineering-org) (from "Inside the AI-Native Engineering Org")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/ffe256ba71a3d5017d4ff5d033858092/thumbnails/thumbnail.jpg?time=242411ms
- [Managing compute costs and AI model routing](https://www.wearedevelopers.com/videos/100328-the-limits-of-llms-in-real-world-applications) (from "The Limits of LLMs in Real-World Applications")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/8b7fd6e0df35abc7aa37933d51077adb/thumbnails/thumbnail.jpg?time=325440ms
- [Optimizing AI infrastructure from applications to GPU kernels](https://www.wearedevelopers.com/videos/100536-how-linkedin-turns-ai-breakthroughs-into-member-and-customer-value) (from "How LinkedIn Turns AI Breakthroughs into member and customer value")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/6fa8e5cff519be48b88cc6c5b8564254/thumbnails/thumbnail.jpg?time=282740ms
- [Prioritizing developer user experience over raw model parameters](https://www.wearedevelopers.com/videos/100256-can-this-elephant-dance-ibm-bob-and-the-future-of-ai-first-software-development) (from "Can This Elephant Dance? IBM Bob and the Future of AI-First Software Development")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/d9f8cc0d9bbcac0ba21727a7959aa947/thumbnails/thumbnail.jpg?time=65750ms
- [Understanding core parameters and mechanics of large language models](https://www.wearedevelopers.com/videos/1512-building-ai-applications-with-langchain-and-node-js) (from "Building AI Applications with LangChain and Node.js")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/7f71af512c5d50d581e626e3361b8deb/thumbnails/thumbnail.jpg?time=57865ms
- [Balancing scale and architecture in artificial intelligence development](https://www.wearedevelopers.com/videos/100476-making-science-larger-not-just-faster) (from "Making Science Larger, not just Faster")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/507e95c8e19f2f31b65dfb93a75cf9f9/thumbnails/thumbnail.jpg?time=191282ms

## Related Articles

- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [Infer Once, Run Everywhere: Zero-Token Architecture](https://www.wearedevelopers.com/magazine/783-infer-once-run-everywhere-zero-token-architecture)

## Related Jobs

- [Engineering Manager, Google Cloud AI/ML](https://www.wearedevelopers.com/jobs/ext/3530886-engineering-manager-google-cloud-ai-ml) at **PwC**
- [Engineering Manager, Google Cloud AI/ML](https://www.wearedevelopers.com/jobs/ext/4292533-engineering-manager-google-cloud-ai-ml) at **PwC**
- [Engineering Manager, Google Cloud AI/ML](https://www.wearedevelopers.com/jobs/ext/4319970-engineering-manager-google-cloud-ai-ml) at **PwC**
- [Engineering Manager, Google Cloud AI/ML](https://www.wearedevelopers.com/jobs/ext/4451516-engineering-manager-google-cloud-ai-ml) at **PwC**
- [Engineering Manager, Google Cloud AI/ML](https://www.wearedevelopers.com/jobs/ext/4313196-engineering-manager-google-cloud-ai-ml) at **PwC**
- [Engineering Manager, Google Cloud AI/ML](https://www.wearedevelopers.com/jobs/ext/4291394-engineering-manager-google-cloud-ai-ml) at **PwC**