WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2025 • Aug 20, 2025 • Session details

Unveiling the Magic: Scaling Large Language Models to Serve Millions

Patrick Koss

Ditch the 20-minute cold starts. Treat self-hosted LLMs as compute-heavy REST APIs. Learn how NFS, Pingora, and intelligent rate-limiting seamlessly scale your infrastructure to serve millions.

Unveiling the Magic: Scaling Large Language Models to Serve Millions thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

2:16 min

Best practices for utilizing large language models in production

Stan Girard Stan Girard · World Congress 2024

2:39 min

Open source tools for running and scaling models

Cedric Clyburn Cedric Clyburn +1 · World Congress 2025

3:14 min

Balancing performance and costs with custom language models

Jemiah Sius Jemiah Sius · World Congress 2024

2:10 min

Evaluating hosting options for large language models

Michal Salanci Michal Salanci · World Congress 2026 North America

4:59 min

Optimizing server architecture for scalable large language model operations

Calvin Seward Calvin Seward · Europe 2026 Virtual

5:25 min

Addressing core challenges in large language model deployments

Vijay Krishan Gupta +1 · LIVE

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail
24:33

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing · AWS

Behind the Code: How Women Are Powering the Future of AI thumbnail
30:28

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel +2

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail
28:12

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe +3

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail
28:19

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler +3

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail
28:09

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher +3

AI Sovereignty: What Does It Take? thumbnail
29:13

AI Sovereignty: What Does It Take?

Boris Hecker +3

Upcoming sessions on this topic

Open session

Bypassing the GPU Tax: Smart Layer-7 Routing with Small Language Models (SLMs)

  • Sagar Utekar

    CrowdStrike

    SRE

Open session

How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design

  • Purnima Kumari

    DigitalOCean

    Sr. Solutions Architect II

Open session

Recursive Language Models: Scaling infinitely beyond the Context Window

  • Jayita Bhattacharyya

    Domyn

    Member of Technical Staff

Open session

Decision models for developers who use LLMs

  • Anurag Bhandari

    antelligent

    CTO

Open session

Your AI Agents Are Bleeding Tokens: Building a Runtime Cost Control Plane for LLMs

  • Aakriti Aggarwal

    IBM Research

    Research Engineer

  • Bhavishya Pandit

    66degrees

    Leading AI Innovation

Open session

Enterprise AI Memory at Scale

  • Aditya Joshi

    Walmart

    Senior Software Engineer

Related videos

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization thumbnail
26:27

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization

Legare Kerrison +1

World Congress North America · 2026

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl · Microsoft

World Congress Europe · 2026

Self-Hosted LLMs: From Zero to Inference thumbnail
30:04

Self-Hosted LLMs: From Zero to Inference

Cedric Clyburn +1

World Congress · 2025

The New Bottleneck in Software Development thumbnail
25:38

The New Bottleneck in Software Development

Aditya Jayaprakash · Blacksmith

World Congress North America · 2026

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow thumbnail
34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

LIVE · Oct 2024

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

World Congress · 2024

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.