> Markdown version of [/videos/1619-unveiling-the-magic-scaling-large-language-models-to-serve-millions?t=110](https://www.wearedevelopers.com/videos/1619-unveiling-the-magic-scaling-large-language-models-to-serve-millions?t=110). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Unveiling the Magic: Scaling Large Language Models to Serve Millions

Ditch the 20-minute cold starts. Treat self-hosted LLMs as compute-heavy REST APIs. Learn how NFS, Pingora, and intelligent rate-limiting seamlessly scale your infrastructure to serve millions.

- **Speakers:** [Patrick Koss](https://www.wearedevelopers.com/@patrick-koss)
- **Event:** World Congress 2025
- **Published:** August 20, 2025
- **Duration:** 31:25
- **URL:** https://www.wearedevelopers.com/videos/1619-unveiling-the-magic-scaling-large-language-models-to-serve-millions
- **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/928f635e1b94b6597c98fc80eebe0e2f/thumbnails/thumbnail.jpg?time=112875ms

## Access

Playback and chapters for this video are available with a Free account.

## Related Moments

- [Best practices for utilizing large language models in production](https://www.wearedevelopers.com/videos/1130-chatbots-are-going-to-destroy-infrastructures-and-your-cloud-bills) (from "Chatbots are going to destroy infrastructures and your cloud bills")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/d31722d9c1008a621694adb4597b7c75/thumbnails/thumbnail.jpg?time=62435ms
- [Open source tools for running and scaling models](https://www.wearedevelopers.com/videos/1597-self-hosted-llms-from-zero-to-inference) (from "Self-Hosted LLMs: From Zero to Inference")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/59dec89bb50d737dca452eccbbe82073/thumbnails/thumbnail.jpg?time=161035ms
- [Balancing performance and costs with custom language models](https://www.wearedevelopers.com/videos/994-mastering-ai-driven-problem-solving-in-engineering-with-observability) (from "Mastering AI-Driven Problem Solving in Engineering with Observability")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/a30a4d87ea3e9010ba37c13558f88140/thumbnails/thumbnail.jpg?time=354275ms
- [Evaluating hosting options for large language models](https://www.wearedevelopers.com/videos/100584-run-your-agents-in-kubernetes-build-once-deploy-anywhere-but-really) (from "Run your agents in Kubernetes: Build once, deploy anywhere. But really?")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/5abcf6a64d9b1a26a0f4775190f6f684/thumbnails/thumbnail.jpg?time=134274ms
- [Optimizing server architecture for scalable large language model operations](https://www.wearedevelopers.com/videos/2088-plan-to-link-your-llm-to-your-production-database-what-could-possibly-go-wrong) (from "Plan to link your LLM to your production database? What could possibly go wrong?")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/7154adbe5cfa76750c41e39cdf821ada/thumbnails/thumbnail.jpg?time=294900ms
- [Addressing core challenges in large language model deployments](https://www.wearedevelopers.com/videos/899-creating-industry-ready-solutions-with-llm-models) (from "Creating Industry ready solutions with LLM Models")
  - **Thumbnail URL:** https://customer-goifxnzhrq3d0d54.cloudflarestream.com/c1137aae2e3d45e2d96a87f795887129/thumbnails/thumbnail.jpg?time=111245ms

## Related Articles

- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [The Best Large Language Models on The Market](https://www.wearedevelopers.com/magazine/319-the-best-large-language-models-on-the-market)
- [Infer Once, Run Everywhere: Zero-Token Architecture](https://www.wearedevelopers.com/magazine/783-infer-once-run-everywhere-zero-token-architecture)

## Related Jobs

- [LLM Training Engineer](https://www.wearedevelopers.com/jobs/48420-llm-training-engineer) at **Sciforium**
- [LLM Dataset Engineer](https://www.wearedevelopers.com/jobs/48419-llm-dataset-engineer) at **Sciforium**
- [Lead Software Engineer, Model Serving Platform](https://www.wearedevelopers.com/jobs/48413-lead-software-engineer-model-serving-platform) at **Sciforium**
- [Model Implementation Engineer](https://www.wearedevelopers.com/jobs/48421-model-implementation-engineer) at **Sciforium**
- [Senior AI/ML Engineer](https://www.wearedevelopers.com/jobs/48352-senior-ai-ml-engineer) at **PagerDuty**
- [Senior AI Serving Engineer, Backend](https://www.wearedevelopers.com/jobs/48414-senior-ai-serving-engineer-backend) at **Sciforium**