WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Oct 13, 2021

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing

Lina Weichbrodt

Is your machine learning model technically functional but quietly damaging your business? Discover how to spot a brain-dead AI using your existing observability stack to prevent silent degradation.

Pause
Mute Enter Fullscreen
#1 about 2 min

Defining success and business requirements in machine learning

Since vague business requirements often derail machine learning projects, developers must establish precise performance criteria upfront.

#2 about 2 min

Example business case for predicting loan application rejections

Generating internal rejection predictions saves businesses from incurring high monthly API processing costs via external agencies.

#3 about 5 min

Choosing precision and recall metrics for loan optimization

To balance financial savings with accuracy, teams adjust prediction thresholds across standard precision and recall metrics.

#4 about 2 min

Applying gradient boosted trees for tabular data training

Since tabular database columns defeat many deep learning approaches, ensemble algorithms provide vastly superior training speed.

#5 about 3 min

Selecting monitoring infrastructure stacks for machine learning operations

Instead of adopting complex proprietary platforms, introductory machine learning teams can repurpose existing engineering dashboard tools.

#6 about 6 min

Monitoring correct model outcomes in delayed production environments

When automated decisions obscure natural outcomes, deploying an untouched traffic slice provides necessary baseline performance metrics.

#7 about 3 min

Translating stakeholder concerns into actionable production monitoring signals

To bridge the gap with non-technical partners, developers translate explicit worst-case fears into monitored production alerts.

#8 about 4 min

Tracking model response distributions when true validations fail

Because ground truth labels arrive late, tracking statistical distance across output distributions reveals silent inference failures.

#9 about 3 min

Implementing common sense quality heuristics for model outputs

When deep statistical validation is impossible, human-understandable proxy metrics immediately highlight degraded algorithmic user experiences.

#10 about 2 min

Identifying pipeline discrepancies through live input feature monitoring

To catch hidden transformations, comparing offline training batches against live API traffic prevents silent feature engineering regressions.

#11 about 4 min

Differentiating model quality metrics from incident detection indicators

By prioritizing endpoint impacts over mathematical accuracy, teams surface critical operational failures before experiencing complete system degradation.

#12 about 16 min

Audience questions on practical machine learning operational strategies

Addressing real-world constraints clarifies delayed feedback management, ideal modeling techniques, and optimal pathways into data engineering.

Matching moments

4:17 min

Overcoming limitations in current machine learning monitoring tools

David Mosen · World Congress 2021

4:01 min

Transitioning artificial intelligence into operational business environments

Stefan Donsa Stefan Donsa +1 · LIVE

1:05 min

Implementing monitoring and observability for AI software deployments

Alejandro Saucedo Alejandro Saucedo · World Congress 2025

2:24 min

Monitoring AI model quality and execution performance

Jemiah Sius Jemiah Sius +1 · World Congress 2026 Europe

3:08 min

Maintaining and monitoring machine learning models in production

Natalie Pistunovich · LIVE

4:20 min

Designing architectures for automated machine learning lifecycles

David Mosen · World Congress 2021

More from WeAreDevelopers LIVE

Fake or News - Claude's DNA Discovery, Quick Cloning & Crime on Insta thumbnail
6:07
1 day ago

Fake or News - Claude's DNA Discovery, Quick Cloning & Crime on Insta

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40
1 day ago

WeAreDevelopers LIVE - Playing in Production

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
15 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
22 days ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
24 days ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
29 days ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Related videos

Deployed ML models need your feedback too thumbnail
37:24
World Congress 2021 • over 5 years ago

Deployed ML models need your feedback too

David Mosen

Why LLMs Need Observability and How to Do It thumbnail
32:41
Europe 2026 Virtual • 3 months ago

Why LLMs Need Observability and How to Do It

Rahul Gupta

DevOps for Machine Learning thumbnail
44:40
World Congress 2021 • over 5 years ago

DevOps for Machine Learning

Hauke Brammer

How AI Models Get Smarter thumbnail
35:16
World Congress 2025 • about 1 year ago

How AI Models Get Smarter

Ankit Patel

Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source thumbnail
29:50
World Congress 2026 Europe • 3 months ago

Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source

Viktoria Semaan

Unlocking the AI Black Box: Building Trust in the Era of Agentic Production thumbnail
27:40
World Congress 2026 Europe • 3 months ago

Unlocking the AI Black Box: Building Trust in the Era of Agentic Production

Jemiah Sius, Harry Kimpel

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.