> Markdown version of [/videos/1014-machine-learning-101-where-to-begin?t=458](https://www.wearedevelopers.com/videos/1014-machine-learning-101-where-to-begin?t=458). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Machine learning 101: Where to begin?

Are you using machine learning just for the hype? Learn the foundational Python workflows, preprocessing techniques, and essential algorithms needed to confidently build your first model.

- **Speakers:** [Lutske van der Meer](https://www.wearedevelopers.com/@lutske-de-leeuw)
- **Event:** World Congress 2024
- **Published:** August 20, 2024
- **Duration:** 26:25
- **URL:** https://www.wearedevelopers.com/videos/1014-machine-learning-101-where-to-begin

## Summary

Using the relatable premise of engineering a smart cat feeder, this introductory guide explores the foundational concepts and practical workflows required to build machine learning applications. Before writing any code, developers must clearly define their specific problem and ensure machine learning is actually necessary. Implementing ML solely for the hype is a common pitfall; if simple business rules can solve the challenge, developers should default to those to avoid the steep costs and time commitments associated with gathering extensive datasets and securing domain experts for data labeling.

Once a valid use case is established, the engineering pipeline relies heavily on the Python data science ecosystem, utilizing tools like Pandas, Scikit-learn, and Jupyter. Developers are encouraged to leverage existing public datasets from platforms like Kaggle rather than starting from scratch. Raw data rarely arrives fully clean, requiring rigorous preprocessing techniques such as dropping missing values, utilizing label or one-hot encoding to translate text categories into performant numerical data, and applying data augmentation to artificially expand limited image sets without causing the model to overfit to specific environments.

The training lifecycle dictates splitting data into training, validation, and testing sets—typically an 80/10/10 ratio—to iteratively teach and evaluate the model. The choice of algorithm depends heavily on the data structure, ranging from linear and logistic regression to decision trees and random forests. Finally, model evaluation dictates deployment readiness. By analyzing tools like a confusion matrix, developers can consciously weigh the domain-specific risks of false positives versus false negatives, while utilizing metrics like Mean Squared Error to measure baseline prediction accuracy and iteratively improve models by capping analytical outliers.

**Keywords:** machine learning fundamentals, python data science ecosystem, scikit-learn model training, kaggle dataset integration, supervised learning algorithms, reinforcement learning structures, data preprocessing pipelines, label and one-hot encoding, image data augmentation, train validation test splitting, linear and logistic regression, decision tree classifiers, random forest algorithms, preventing model overfitting, confusion matrix evaluation, mean squared error metric, handling regression outliers

## Chapters

1. **Introduction to machine learning with a practical cat problem** (00:03) — Over-engineering a pet food dispenser illustrates the basic motivation for exploring machine learning.
1. **Distinguishing between AI, machine learning, and deep learning** (02:06) — Machine learning teaches computers without explicit programming while deep learning relies on neural networks to process specific tasks.
1. **Understanding supervised, unsupervised, and reinforcement learning models** (03:27) — Different learning types vary from labeling known data to categorizing raw data and rewarding actions in dynamic environments.
1. **Evaluating whether machine learning is necessary for business goals** (05:33) — Relying on existing business rules often proves more cost-effective than investing extensive time and domain expertise into AI hype.
1. **Setting up Python libraries and sourcing initial datasets** (07:38) — Essential data science libraries like scikit-learn and pandas enable processing available datasets from platforms like Kaggle.
1. **Downloading and inspecting data frames via the Kaggle API** (10:02) — Authenticating with Kaggle to download a raw target dataset reveals common formatting inconsistencies and missing values requiring cleanup.
1. **Cleaning missing values and expanding datasets with data augmentation** (11:24) — Applying label encoders and generating synthetic images prevents inadequate training sizes while requiring care to avoid over-fitting.
1. **Splitting data for training, validation, and testing stages** (14:25) — Dividing datasets securely allows iterative learning and final independent accuracy validations before real-world deployment.
1. **Exploring common regression and classification machine learning algorithms** (16:14) — Foundational techniques span from fitting simple linear regression lines to using random forest classifiers for majority-ruled category predictions.
1. **Evaluating model accuracy with confusion matrices and mean squared error** (18:46) — Visualizing over-fitting limits and calculating statistical errors helps ground raw prediction values into tangible business impacts.
1. **Refining models by managing outliers and sourcing more data** (23:14) — Strategies like trimming extreme outliers via winsorizing actively enhance accuracy when preliminary outputs miss intended targets.
1. **Real life recommendation systems and final project conclusions** (24:31) — Examining proven recommendation setups anchors core lessons around thoughtful data collection and strict model verification.

## Related Moments

- [Introduction to prototyping and building practical AI applications](https://www.wearedevelopers.com/videos/1010-bringing-the-power-of-ai-to-your-application) (from "Bringing the power of AI to your application.")
- [Navigating the complexities of machine learning model lifecycles](https://www.wearedevelopers.com/videos/1657-dataforce-studio) (from "DataForce Studio")
- [Audience questions on practical machine learning operational strategies](https://www.wearedevelopers.com/videos/262-is-my-ai-alive-but-brain-dead-how-monitoring-can-tell-you-if-your-machine-learning-stack-is-still-performing) (from "Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing")
- [Core stages of training and managing machine learning models](https://www.wearedevelopers.com/videos/157-intelligent-automation-using-machine-learning) (from "Intelligent Automation using Machine Learning")
- [Familiarizing with machine learning and neural network basics](https://www.wearedevelopers.com/videos/227-uncertainty-estimation-of-neural-networks) (from "Uncertainty Estimation of Neural Networks")
- [Understanding the machine learning building workflow](https://www.wearedevelopers.com/videos/272-machine-learning-in-ml-net) (from "Machine Learning in ML.NET")

## Related Articles

- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [How to start an AI project for a good cause and boost your career](https://www.wearedevelopers.com/magazine/15-how-to-start-an-ai-project-for-a-good-cause-and-boost-your-career)

## Related Jobs

- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [Principal Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1706410-principal-machine-learning-engineer) at **Almedia**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1355348-machine-learning-engineer) at **TWILIO**
- [Data Scientist](https://www.wearedevelopers.com/jobs/ext/1351648-data-scientist) at **Almedia**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1377841-machine-learning-engineer) at **Almedia**
- [Staff, Machine Learning Engineer (L4)](https://www.wearedevelopers.com/jobs/ext/1202639-staff-machine-learning-engineer-l4) at **Twilio**