Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization
September 24, 2026
16:10
–
16:40
·
30
min
Stage 6
What this session covers
Running large language models at scale can get expensive fast, but the right optimizations can cut latency and GPU costs dramatically.
We’ll walk through how to serve models efficiently using vLLM, an open-source, high-performance inference engine. Then and generate and test quantized models, expose them through vLLM’s OpenAI-compatible API, and tune runtime flags to balance throughput, latency, and accuracy on different GPUs.
We’ll benchmark performance live, inspect token-throughput metrics, and discuss real-world deployment trade-offs.
World Congress 2026 North America
Related talks
- Understanding LLM Architectures: Inside the Design of Modern Models
Jofia Jose Prakash · Humanity + AI, Inc
Thu 24 Sep · Stage 3
- Anatomy of an AI Request: Where Latency and Cost Are Really Born
Dan Fu · Together AI
Thu 24 Sep · Stage 1
- KV Cache Is Not About Speed: It's About Surviving Inference Costs
David vonThenen · NetApp
Thu 24 Sep · Stage 2
- No Single Model to Rule Them All: Building Resilient AI Agents Across Open & Closed LLMs
Emmanuel Acheampong · Crusoe AI
Thu 24 Sep · Stage 6