> Markdown version of [/videos/1654-distil-labs-small-model-training-made-simple](https://www.wearedevelopers.com/videos/1654-distil-labs-small-model-training-made-simple). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# distil labs – small model training, made simple

Upgrading vanilla small models used to require specialized machine learning scientists and expensive infrastructure. Now, standard engineering teams can train highly accurate, on-premise expert models in just eight hours.

- **Speakers:** [Selim Nowicki](https://www.wearedevelopers.com/@selim-nowicki)
- **Event:** World Congress 2025
- **Published:** August 20, 2025
- **Duration:** 5:13
- **URL:** https://www.wearedevelopers.com/videos/1654-distil-labs-small-model-training-made-simple

## Summary

Although large foundational models are the default choice for general AI tasks, they present significant hurdles for on-device processing, high-throughput efficiency, and strict data privacy requirements. Transitioning to small models solves on-premise deployment and latency challenges, but vanilla small models typically suffer from poor accuracy out of the box. Upgrading them into highly accurate expert models historically demands specialized machine learning scientists, expensive evaluation infrastructure, and tens of thousands of manually labeled data points.

Distil Labs eliminates these bottlenecks by providing a developer-focused platform that builds small expert models using automated knowledge distillation. Teams only need to supply a task prompt, 50 labeled data points, and any existing unstructured organizational data. The platform heavily utilizes large language models to generate and validate synthetic training data, cleverly constructing a robust educational curriculum that rapidly upskills the smaller model without requiring immense manual labeling efforts.

Within just eight hours, developers receive a fully fine-tuned small model that matches large model accuracy for a targeted single task. By producing models 50 to 100 times smaller than standard foundational counterparts, organizations can unlock entirely new application design patterns while avoiding costly recurring inference fees. This automated pipeline ultimately empowers standard engineering teams to deploy private, on-premise AI capabilities completely independently, bypassing the expensive traditional requirement of hiring an entire machine learning department.

**Keywords:** small expert models, knowledge distillation, automated fine-tuning, on-device AI processing, on-premise AI deployment, LLM synthetic data generation, AI latency optimization, high-throughput inference costs, foundational model alternatives, task-specific AI training, data privacy compliance, model size reduction, machine learning automation, unstructured AI data integration

## Chapters

1. **The case for deploying small foundational models** (00:04) — Small models provide critical advantages for local deployment, data privacy, and latency-sensitive applications.
1. **The challenge of specializing vanilla small models** (01:53) — Specializing out-of-the-box small models requires significant machine learning expertise and extensive labeled data.
1. **Automating expert model creation with minimal data** (02:53) — Automating the training process empowers developers to build highly accurate expert models using simple prompts.
1. **Applying synthetic data generation and knowledge distillation** (03:34) — Generating synthetic data with large language models automates the fine-tuning process through knowledge distillation.

## Related Moments

- [Training small AI models on secure private data](https://www.wearedevelopers.com/videos/100253-ai-in-high-stakes-industries-lessons-learned) (from "AI in High-Stakes Industries: Lessons Learned")
- [Distilling cloud AI capabilities into local device models](https://www.wearedevelopers.com/videos/1354-google-gemma-and-open-source-ai-models-clement-farabet) (from "Google Gemma and Open Source AI Models - Clement Farabet")
- [Achieving high inference performance in smaller model sizes](https://www.wearedevelopers.com/videos/1332-google-gemini-open-source-and-deep-thinking-models-sam-witteveen) (from "Google Gemini: Open Source and Deep Thinking Models - Sam Witteveen")
- [The case for fine-tuning small models in agentic AI](https://www.wearedevelopers.com/videos/100352-fine-tuning-small-language-models-for-agentic-ai) (from "Fine-Tuning Small Language Models for Agentic AI")
- [Reducing model size using knowledge distillation with teacher-student models](https://www.wearedevelopers.com/videos/158-serverless-deployment-of-large-nlp-models) (from "Serverless deployment of (large) NLP models ")
- [Adopting fine-tuned task oriented AI models](https://www.wearedevelopers.com/videos/1743-wearedevelopers-live-ai-vs-the-web-ai-in-browsers) (from "WeAreDevelopers LIVE – AI vs the Web & AI in Browsers")

## Related Articles

- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)

## Related Jobs

- [Data Scientist](https://www.wearedevelopers.com/jobs/ext/1351648-data-scientist) at **Almedia**
- [Staff, Machine Learning Engineer (L4)](https://www.wearedevelopers.com/jobs/ext/1202639-staff-machine-learning-engineer-l4) at **Twilio**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1355348-machine-learning-engineer) at **TWILIO**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/2013734-machine-learning-engineer) at **GitHub**