WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2025 • Aug 20, 2025 • Session details

Unveiling the Magic: Scaling Large Language Models to Serve Millions

Patrick Koss

Ditch the 20-minute cold starts. Treat self-hosted LLMs as compute-heavy REST APIs. Learn how NFS, Pingora, and intelligent rate-limiting seamlessly scale your infrastructure to serve millions.

Unveiling the Magic: Scaling Large Language Models to Serve Millions thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

2:16 min

Best practices for utilizing large language models in production

Stan Girard Stan Girard · World Congress 2024

2:39 min

Open source tools for running and scaling models

Cedric Clyburn Cedric Clyburn +1 · World Congress 2025

3:14 min

Balancing performance and costs with custom language models

Jemiah Sius Jemiah Sius · World Congress 2024

2:10 min

Evaluating hosting options for large language models

Michal Salanci Michal Salanci · World Congress 2026 North America

4:59 min

Optimizing server architecture for scalable large language model operations

Calvin Seward Calvin Seward · Europe 2026 Virtual

5:25 min

Addressing core challenges in large language model deployments

Vijay Krishan Gupta +1 · LIVE

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail
24:33
12 months ago

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing

Behind the Code: How Women Are Powering the Future of AI thumbnail
30:28
about 1 year ago

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel, Laura Moritz, Madalina Florean

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail
28:12
about 1 year ago

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe, Konstantin Vinogradov, Nils Eiteneyer, Stephan Wirris

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail
28:19
about 1 year ago

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler, Jasmin Kaiser, Jeff Hausmann, Sonja Alvarez

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail
28:09
about 1 year ago

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher, Jan Oberhauser, Keno Dreßel, Mirko Novakovic

AI Sovereignty: What Does It Take? thumbnail
29:13
about 1 year ago

AI Sovereignty: What Does It Take?

Boris Hecker, Clemens Wasner, Robin Hermann, Sonja Alvarez

Upcoming sessions on this topic

Open session

How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design

  • Purnima Kumari

    DigitalOCean

    Sr. Solutions Architect II

Open session

Building Elastic Cloud Serverless on Kubernetes: Lessons from Operating at Cloud Scale

  • Someshwaran Mohan Kumar

    Elastic

    Senior Developer Advocate

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Open session

Shipping MCP to Real Users: Scaling, Auth, and the PII Leak You Didn't See Coming

  • Hadar Geva

    Myop

    CTO & Co-founder

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Open session

How We Cut Our API's p99 Latency from Minutes to Under a Second

  • Deepak Agrawal

    Atlassian

    Principal Software Engineer

Related videos

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization thumbnail
26:27
World Congress 2026 North America • 29 days ago

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization

Legare Kerrison, Cedric Clyburn

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40
World Congress 2026 Europe • 4 months ago

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl

Self-Hosted LLMs: From Zero to Inference thumbnail
30:04
World Congress 2025 • about 1 year ago

Self-Hosted LLMs: From Zero to Inference

Cedric Clyburn, Roberto Carratalá

The New Bottleneck in Software Development thumbnail
25:38
World Congress 2026 North America • 29 days ago

The New Bottleneck in Software Development

Aditya Jayaprakash

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow thumbnail
34:21
LIVE • about 2 years ago

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01
World Congress 2024 • about 2 years ago

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.