WeAreDevelopers LIVE • Nov 17, 2020

Debugging Machine Learning Code

Svetlin Penkov

Are you mistaking basic code errors for fundamental flaws in your neural network? Discover how 3D interactive debuggers and automated assertions instantly catch silent training failures.

Pause

Mute

Enter Fullscreen

#1 about 3 min

Speaker background and the need for robust models

Background of the speaker and the necessity of highly robust machine learning code in autonomous driving.

#2 about 5 min

Understanding computation and data in machine learning

How machine learning models function as computational graphs executing mathematical operations across multi-dimensional data arrays.

#3 about 6 min

The two interconnected loops of model development

The distinction between analyzing structural semantic flaws and handling purely mathematical runtime performance issues.

#4 about 12 min

Limitations of traditional machine learning debugging approaches

Why mental verification, terminal printouts, simple plotting, and custom dashboards fail to scale for complex architectures.

#5 about 18 min

Introducing the FMRI debugger for 3D computational insights

How the FMRI debugger visually maps out code, data, gradients, and automated assertions natively in a fully interactive spatial interface.

#6 about 7 min

Interactive demo of visual tensor inspection and assertions

Resolving vanishing gradients and invalid probability inputs by visually inspecting multi-dimensional structures alongside automated mathematical tests.

#7 about 3 min

Visualizing large-scale model architectures like ResNet-101

Expanding deep individual computational layers and verifying feature maps within large open-source image recognition architectures.

Matching moments

1:31 min

Debugging distributed systems with artificial intelligence tools

5:01 min

Leveraging large language models for code optimization and development

Stephan Gillich

Stephan Gillich +3 · WWC 2024

2:41 min

Leveraging AI tooling for accelerated learning and productivity

Dan Mindru · Coffee With Developers

2:16 min

Interpreting complex model behavior without ground truth data

Kilian Kluge +1 · WWC 2022

1:51 min

Debugging costs of runtime data shape mismatches

Violina Popova

Violina Popova · Europe 2026 Virtual

3:01 min

Balancing artificial intelligence tools with foundational software engineering skills

Tim Ruscica · Coffee With Developers

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail

about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

World Congress 2026 North America

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan

World Congress 2026 North America

There's no dark factory without better software verifiers

Dexter Horthy

Co-Founder, HumanLayer

Dexter Horthy

World Congress 2026 North America

AI That Argues With Itself: Building Self-Debating Systems That Catch Their Own Bugs

Shreya Singhal

AI Applied Scientist at Claritev

Shreya Singhal

World Congress 2026 North America

Closing the Visibility Gap: Lessons from Safety Critical Agentic Systems

Vivek Pandit

Principal Engineer at Cadence

Vivek Pandit

World Congress 2026 North America

Engineering the Pivot: How Creative Strategy Solves the Hard Problems of AI Accuracy and Scale

Shruti Tiwari

AI/ML product manager, Dell

Shruti Tiwari

World Congress 2026 North America

Trust, But Verify: Continuous GPU Validation at Scale

Kyle Bell

VP of AI @ TensorWave

Kyle Bell

Related videos

Getting Started with Machine Learning thumbnail

LIVE • almost 5 years ago

Getting Started with Machine Learning

Alexandra Waldherr

Why LLMs Need Observability and How to Do It thumbnail

Europe 2026 Virtual • about 1 month ago

Why LLMs Need Observability and How to Do It

Rahul Gupta

Talk to the Duck - Secrets of the Debugging Masters thumbnail

WWC 2022 • about 4 years ago

Talk to the Duck - Secrets of the Debugging Masters

Shai Almog

Debugging Unveiled: Exploring Debugger Internals and Hidden Gems thumbnail

LIVE • over 2 years ago

Debugging Unveiled: Exploring Debugger Internals and Hidden Gems

Johannes Bechberger

Building Your Own Classification Model with JavaScript - Coffee with Developers - Carly Richmond thumbnail

Coffee With Developers • over 1 year ago

Building Your Own Classification Model with JavaScript - Coffee with Developers - Carly Richmond

Carly Richmnd

A hundred ways to wreck your AI - the (in)security of machine learning systems thumbnail

WWC 2023 • almost 3 years ago

A hundred ways to wreck your AI - the (in)security of machine learning systems

Balázs Kiss

Related articles

DC

Daniel Cranney

AI-Powered Debugging: The Future of Fixing Your Code

The reality of being a developer is that we often spend more time fixing code that actually writing new code. Debugging is such a core and time-consuming part of software development, it’s no surpr...

over 1 year ago

A page of code with a magnifying glass over it, highlighting the word BUG in red text.

BB

Benedikt Bischof

MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production

Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Bas Geerdink who gave advice on MLOps.‍About the speaker:‍Bas is a programmer, scientist, a...

almost 4 years ago

MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production

BB

Benedikt Bischof

MLOps And AI Driven Development

Welcome to this issue of the WeAreDevelopers Dev Talk Recap series. This article recaps an interesting talk by Natalie Pistunovic who spoke about the development of AI and MLOps. What you will lear...

over 4 years ago

MLOps And AI Driven Development

BB

Benedikt Bischof

MLOps – What’s the deal behind it?

Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine...

almost 4 years ago

MLOps – What’s the deal behind it?

From learning to earning

Jobs that call for the skills explored in this talk.

about 2 months ago

Machine Learning Engineer

Experienced Remote Spacy Pytorch Chatbots

Machine Learning Engineer

Barcelona, Spain

€25k–60k Remote Spacy Twilio Pytorch

about 2 months ago • Verified

Senior AI Agent Software Engineer (Go, Python) (m/f/x)

Vienna, Austria

Experienced €61k–76k Remote Go Python AI Frameworks

Staff Software Engineer, Copilot Experiences

Remote Ruby Docker Sentry

Data Scientist

Berlin, Germany

€80k–200k Airflow Terraform Kubernetes

Machine Learning Engineer

Oberkochen, Germany

Git Bicep Github