When your machine learning models work in sensitive areas like loan application prediction, you better be sure that they work correctly and you are alerted in real-time if there is an issue. But how? We will walk you through a real use case and: * suggest which monitoring signals to prioritize * explain the difference between metrics for model training and metrics for model monitoring * show how machine learning monitoring can be implemented with a traditional software monitoring stack like Grafana and Prometheus