> Markdown version of [/videos/227-uncertainty-estimation-of-neural-networks](https://www.wearedevelopers.com/videos/227-uncertainty-estimation-of-neural-networks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Uncertainty Estimation of Neural Networks

Why do neural networks make dangerously overconfident predictions in novel situations? Learn to calculate true model uncertainty to build safer, self-aware autonomous systems.

- **Speakers:** Tillman Radmer, Fabian Hüger, Nico Schmidt
- **Event:** World Congress 2021
- **Published:** July 1, 2021
- **Duration:** 1:00:28
- **URL:** https://www.wearedevelopers.com/videos/227-uncertainty-estimation-of-neural-networks

## Summary

Deep neural networks (DNNs) deployed in safety-critical environments like autonomous driving struggle to safely navigate rare events. Because traditional softmax scores are poor predictors of actual confidence, models frequently exhibit dangerous overconfidence when encountering novel situations. Resolving this brittleness requires distinguishing between aleatoric uncertainty—inherent measurement noise, such as shifting pixels around a sharp visual edge—and epistemic uncertainty, which stems from knowledge gaps due to lacking training data. By correctly categorizing and evaluating these factors, engineering teams can build models that recognize their own limitations and operational boundaries.

To reliably estimate neural network uncertainty, developers can leverage techniques like alert systems—secondary models trained specifically to predict base model failures—or Monte Carlo (MC) Dropout. Standard MC Dropout processes samples repeatedly with disabled neurons to form a calibrated uncertainty distribution, but it is computationally expensive. However, optimizing this by confining dropout to the final convolutional layer or utilizing a student-teacher knowledge distillation approach allows systems to achieve reliable, constant-time inference suitable for real-time vehicle computing.

Applying robust uncertainty estimation directly enables advanced active learning frameworks and AI safety protocols. In modern automotive "big loop" systems, constantly uploading all sensor data is economically and technically unfeasible since most driving scenarios are entirely routine. Instead, embedded uncertainty triggers act as intelligent data collectors, automatically pruning redundant inputs and curating highly informative "corner cases" to optimize the labeling and retraining pipeline. From a safety verification perspective, uncertainty monitors serve as crucial architectural fallbacks against insufficient generalization; when encountering confusing or out-of-distribution inputs, these monitors can detect erroneous-yet-confident predictions and safely downgrade the system to defensive driving strategies or execute a safe stop.

**Keywords:** uncertainty estimation, aleatoric uncertainty, epistemic uncertainty, deep neural networks, monte carlo dropout, neural network calibration, corner case detection, autonomous driving safety, active learning frameworks, intelligent data collection, knowledge distillation, softmax probability limitations, out-of-distribution inputs, ai safety verification, automotive big loop

## Chapters

1. **Familiarizing with machine learning and neural network basics** (00:02) — Assessing general audience knowledge on foundational machine learning and neural network concepts.
1. **Differentiating between aleatoric and epistemic uncertainty** (03:48) — Analyzing how measurement noise and rare events contribute differently to model uncertainty.
1. **Estimating uncertainty using classification scores and calibration** (07:55) — Standard soft-max scores function poorly as uncertainty predictors unless corrected through output calibration.
1. **Predicting neural network failures with secondary alert models** (10:57) — Secondary alert models offer an inexpensive method to anticipate when a base neural network might fail.
1. **Using dropout sampling to estimate neural network uncertainty** (12:37) — Bayesian approximations implemented via network dropout generate a distribution of predictions to measure system uncertainty.
1. **Identifying automotive applications for uncertainty estimation** (19:48) — Crowdsourced audience input categorizes essential use cases for uncertainty tracking within advanced driver assistance systems.
1. **Managing continuous sensor data with corner case detection** (27:47) — Active learning loops rely on predefined trigger functions to filter informative corner cases from massive sensor data environments.
1. **Improving segmentation models with uncertainty-based data selection** (34:20) — Calculating component-wise uncertainty on semantic segmentation outputs highlights priority edge cases required for model retraining.
1. **Evaluating dataset bias and active learning framework weaknesses** (41:22) — Static datasets and generalized evaluation workflows restrict the overall effectiveness of uncertainty-based data selection frameworks.
1. **Ensuring verifiable artificial intelligence safety in autonomous driving** (45:37) — Recognizing insufficient generalization and unreliable confidence measures is necessary for establishing verifiable neural network safety standards.
1. **Developing safety augmentation processes for neural network deployments** (52:18) — Iterative safety augmentation pipelines and model-driven architectures reduce operational risks within open-world autonomous environments.
1. **Transitioning into the automotive artificial intelligence safety field** (57:26) — Software engineers can transition into artificial intelligence safety by applying proven development practices to analytical systems.

## Related Moments

- [Introduction to safety-critical machine learning in automotive contexts](https://www.wearedevelopers.com/videos/397-what-non-automotive-machine-learning-projects-can-learn-from-automotive-machine-learning-projects) (from "What non-automotive Machine Learning projects can learn from automotive Machine Learning projects")
- [Audience Q&A on autonomous driving models and data](https://www.wearedevelopers.com/videos/519-finding-the-unknown-unknowns-intelligent-data-collection-for-autonomous-driving-development) (from "Finding the unknown unknowns: intelligent data collection for autonomous driving development")
- [Establishing comprehensive safety assurance cases for operational machine learning](https://www.wearedevelopers.com/videos/397-what-non-automotive-machine-learning-projects-can-learn-from-automotive-machine-learning-projects) (from "What non-automotive Machine Learning projects can learn from automotive Machine Learning projects")
- [Addressing participant questions on liability and machine learning](https://www.wearedevelopers.com/videos/374-the-future-of-automotive-mobility-upcoming-e-e-architectures-v2x-and-its-challenges) (from "The future of automotive mobility: Upcoming E/E architectures, V2X and its challenges")
- [Navigating automotive complexity with AI runtime environments](https://www.wearedevelopers.com/videos/198-developing-an-ai-sdk) (from "Developing an AI.SDK")
- [Speaker background and the need for robust models](https://www.wearedevelopers.com/videos/71-debugging-machine-learning-code) (from "Debugging Machine Learning Code")

## Related Articles

- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)
- [WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?](https://www.wearedevelopers.com/magazine/469-wwc24-talk-scott-hanselman-ai-superhero-or-supervillain)

## Related Jobs

- [Data Scientist](https://www.wearedevelopers.com/jobs/ext/1351648-data-scientist) at **Almedia**
- [Staff Software Engineer, Copilot Experiences](https://www.wearedevelopers.com/jobs/ext/164361-staff-software-engineer-copilot-experiences) at **GitHub**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [Principal Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1706410-principal-machine-learning-engineer) at **Almedia**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1377841-machine-learning-engineer) at **Almedia**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**