David Mosen

Deployed ML models need your feedback too

Is your deployed ML model slowly failing in production? Learn why feedback loops are the missing piece for monitoring true performance and preventing model decay.

Deployed ML models need your feedback too
#1about 2 minutes

The role of feedback in the MLOps lifecycle

MLOps extends traditional software engineering by integrating processes from data science and business to ensure deployed models perform correctly.

#2about 2 minutes

Understanding the three pillars of MLOps

MLOps adapts DevOps principles by adding continuous training to continuous integration and delivery, addressing the unique needs of ML models.

#3about 4 minutes

Architecting a mature and automated MLOps pipeline

A mature MLOps architecture automates the entire lifecycle from feature store to prediction service, but requires monitoring to close the loop.

#4about 8 minutes

Exploring the different layers of model monitoring

Effective monitoring covers multiple layers, including infrastructure, data drift, concept drift, model performance, and business KPIs.

#5about 4 minutes

Key characteristics of an effective feedback system

Designing a feedback system requires considering the delay, collection method, and correlation of feedback to predictions.

#6about 4 minutes

Evaluating the state of current monitoring solutions

While many tools exist for monitoring training, live performance monitoring is less mature, with platforms like Google Vertex AI and Seldon Core having limitations.

#7about 8 minutes

Demo of a unified model and business monitoring dashboard

An internal tool demonstrates how to integrate with cloud AI services like AWS Personalize to provide a unified view of model and business metrics.

#8about 5 minutes

Q&A on multi-tenant models and edge deployment

The discussion covers best practices for deploying models to different customers, handling unstructured data, and adapting monitoring concepts for edge devices.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

Why corporate AI adoption lags behind the hype
03:28 MIN

Why corporate AI adoption lags behind the hype

What 2025 Taught Us: A Year-End Special with Hung Lee

Moving beyond headcount to solve business problems
04:27 MIN

Moving beyond headcount to solve business problems

What 2025 Taught Us: A Year-End Special with Hung Lee

Balancing the trade-off between efficiency and resilience
03:38 MIN

Balancing the trade-off between efficiency and resilience

What 2025 Taught Us: A Year-End Special with Hung Lee

Breaking down silos between HR, tech, and business
03:39 MIN

Breaking down silos between HR, tech, and business

What 2025 Taught Us: A Year-End Special with Hung Lee

Automating formal processes risks losing informal human value
03:48 MIN

Automating formal processes risks losing informal human value

What 2025 Taught Us: A Year-End Special with Hung Lee

Rapid-fire thoughts on the future of work
02:44 MIN

Rapid-fire thoughts on the future of work

What 2025 Taught Us: A Year-End Special with Hung Lee

Incentivizing automation with a 'keep what you kill' policy
05:18 MIN

Incentivizing automation with a 'keep what you kill' policy

What 2025 Taught Us: A Year-End Special with Hung Lee

Featured Partners

Related Videos

See all videos
Effective Machine Learning - Managing Complexity with MLOps45:45

Effective Machine Learning - Managing Complexity with MLOps

Simon Stiebellehner

DevOps for Machine Learning44:40

DevOps for Machine Learning

Hauke Brammer

The state of MLOps - machine learning in production at enterprise scale47:19

The state of MLOps - machine learning in production at enterprise scale

Bas Geerdink 

MLOps - What’s the deal behind it?29:24

MLOps - What’s the deal behind it?

Nico Axtmann

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing48:07

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing

Lina Weichbrodt

From Traction to Production: Maturing your GenAIOps step by step25:27

From Traction to Production: Maturing your GenAIOps step by step

Maxim Salnikov

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment25:41

AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment

Joy Chatterjee

Related Articles

View all articles
BB
Benedikt Bischof
MLOps – What’s the deal behind it?
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine learning veteran. Starting back in 2014 he observed ...
MLOps – What’s the deal behind it?
BB
Benedikt Bischof
MLOps And AI Driven Development
Welcome to this issue of the WeAreDevelopers Dev Talk Recap series. This article recaps an interesting talk by Natalie Pistunovic who spoke about the development of AI and MLOps. What you will learn:How the concept of AI became an academic field and ...
MLOps And AI Driven Development

From learning to earning

Jobs that call for the skills explored in this talk.

Data Engineering

Data Engineering

Mirelo AI
Berlin, Germany

Azure
Machine Learning
Amazon Web Services (AWS)