Lukas Kölbl

Anomaly Detection - Using unsupervised Machine Learning for detecting anomalies in customer base

What happens when your labeled data is unusable? See how this team pivoted to an unsupervised model that successfully detected 84% of true customer outliers.

Anomaly Detection - Using unsupervised Machine Learning for detecting anomalies in customer base
#1about 5 minutes

The essential skills of a modern data scientist

A data scientist needs a blend of math, statistics, and technology skills, but business knowledge and communication are the most crucial for success.

#2about 2 minutes

Understanding the real data science project workflow

The majority of a data scientist's time is spent on data cleansing and feature engineering, not just model training, requiring close collaboration with business stakeholders.

#3about 3 minutes

Defining the customer anomaly detection use case

An insurance company sought to automate the detection of customer outliers to improve user experience, moving from a manual, time-consuming process to an unbiased, data-driven one.

#4about 4 minutes

Building the analytical record for the model

The project's core effort involved creating a master data table, or analytical record, which consumed 70% of the time and required shifting from a supervised to an unsupervised approach due to data quality issues.

#5about 3 minutes

Using robust PCA for explainable anomaly detection

A robust Principal Component Analysis (PCA) model was chosen to identify outliers by measuring reconstruction error after dimensionality reduction, offering a simple and explainable solution.

#6about 4 minutes

Analyzing model results and business impact

The model successfully detected 84% of true outliers, as shown by a confusion matrix and a traffic light visualization, significantly improving efficiency over manual processes.

Related jobs
Jobs that call for the skills explored in this talk.

Featured Partners

Related Videos

Detecting Money Laundering with AI29:37

Detecting Money Laundering with AI

Stefan Donsa & Lukas Alber

Overview of Machine Learning in Python57:46

Overview of Machine Learning in Python

Adrian Schmitt

Data Science in Retail58:21

Data Science in Retail

Julian Joseph

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing48:07

Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing

Lina Weichbrodt

How Machine Learning is turning the Automotive Industry upside down26:55

How Machine Learning is turning the Automotive Industry upside down

Jan Zawadzki

Finding the unknown unknowns: intelligent data collection for autonomous driving development19:24

Finding the unknown unknowns: intelligent data collection for autonomous driving development

Liang Yu

Empowering Retail Through Applied Machine Learning22:41

Empowering Retail Through Applied Machine Learning

Christoph Fassbach, Daniel Rohr

Intelligent Automation using Machine Learning2:11:02

Intelligent Automation using Machine Learning

Boris Krumrey, Andreas Palfi & Radu Pruna

From learning to earning

Jobs that call for the skills explored in this talk.