> Markdown version of [/events/world-congress-2025/sessions/907-unveiling-the-magic](https://www.wearedevelopers.com/events/world-congress-2025/sessions/907-unveiling-the-magic). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Unveiling the Magic: Scaling Large Language Models to Serve Millions


- **Date:** Friday, Jul 11, 2025
- **Time:** 15:40–16:10 (30 min)
- **Room:** Stage 7
- **Event:** World Congress 2025
- **Tags:** ai, kubernetes, llms, performance, scalability

## Recording

[Watch recording](https://www.wearedevelopers.com/videos/1619-unveiling-the-magic-scaling-large-language-models-to-serve-millions)

## Description

 Ever wondered what's behind the curtain when you chat with AI like OpenAI's ChatGPT?

At STACKIT, we've developed a model-serving service designed to host chat models, embedding models, and more, with a particular focus on supporting large language models.

In this talk, we'll unveil the challenges and solutions of scaling large language models (LLMs) to serve millions. We'll explore model acquisition, caching, building an OpenAI-compatible server, secure authentication, usage tracking for billing, autoscaling, and hosting multiple models under the domain.

Whether you're a developer, engineer, or just curious, this talk will show you how every token counts in making AI accessible to millions.

## Speaker

### [Patrick Koss](https://www.wearedevelopers.com/@patrick-koss)

STACKIT, Tech Lead


## Related talks at this congress

- [One AI API to Power Them All](https://www.wearedevelopers.com/events/world-congress-2025/sessions/718-one-ai-api-to-power) — Roberto Carratalá
- [How AI Models Get Smarter](https://www.wearedevelopers.com/events/world-congress-2025/sessions/659-how-ai-models-get) — Ankit Patel
- [Exploring LLMs across clouds](https://www.wearedevelopers.com/events/world-congress-2025/sessions/525-exploring-llms) — Tomislav Tipurić
- [Your Next AI Needs 10,000 GPUs. Now What?](https://www.wearedevelopers.com/events/world-congress-2025/sessions/586-your-next-ai-needs) — Anshul Jindal, Martin Piercy