Francis Powlesland & Elena Kotljarova

On the straight and narrow path - How to get cars to drive themselves using reinforcement learning and trajectory optimization

From erratic exploration to a perfectly optimized path in 150 laps. See how a race car learns to drive itself using reinforcement learning.

On the straight and narrow path - How to get cars to drive themselves using reinforcement learning and trajectory optimization
#1about 5 minutes

A novel approach to self-driving cars

This project uses reinforcement learning to enable a car to learn on-the-drive, unlike pre-trained models that rely on static data.

#2about 3 minutes

Setting a baseline with a human driver

A human driver completes three laps on the physical racetrack to establish a benchmark time for the AI to compete against.

#3about 5 minutes

Observing the AI learn across 1500 laps

The AI's driving behavior evolves from random and unstable after 15 laps to smooth and optimized after 150, showing diminishing returns by 1500 laps.

#4about 6 minutes

Understanding the core concepts of reinforcement learning

A recap of the demo's results leads into an explanation of reinforcement learning's core ideas like agents, environments, actions, and maximizing rewards.

#5about 5 minutes

Applying Q-learning with states, actions, and Q-tables

Q-learning uses a table of states and actions to store learned values, making it easy to inspect and update the agent's knowledge.

#6about 4 minutes

Key parameters for tuning the Q-learning algorithm

The algorithm's behavior is controlled by key parameters like the learning rate (alpha), discount factor, and the exploration factor (epsilon).

#7about 1 minute

The technical architecture of the race track demo

The demo integrates PS4 controllers, an Arduino, the Watson IoT platform, a Node.js backend, and a React.js frontend.

#8about 3 minutes

Real-world application with Thyssen Krupp

A collaboration with Thyssen Krupp applies these reinforcement learning concepts to a full-size vehicle to learn and adapt its driving style.

#9about 7 minutes

Q&A on data, constraints, and local optima

The speakers answer audience questions about the importance of data quality, how the car stays on the track, and how the algorithm avoids local optima.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

How AI can create more human moments in HR
03:13 MIN

How AI can create more human moments in HR

The Future of HR Lies in AND – Not in OR

Shifting from talent acquisition to talent architecture
03:28 MIN

Shifting from talent acquisition to talent architecture

The Future of HR Lies in AND – Not in OR

Balancing business, technology, and people for holistic success
06:51 MIN

Balancing business, technology, and people for holistic success

The Future of HR Lies in AND – Not in OR

The importance of a fighting spirit to avoid complacency
06:04 MIN

The importance of a fighting spirit to avoid complacency

The Future of HR Lies in AND – Not in OR

Navigating ambiguity as a core HR competency
04:22 MIN

Navigating ambiguity as a core HR competency

The Future of HR Lies in AND – Not in OR

How the HR function has evolved over three decades
05:10 MIN

How the HR function has evolved over three decades

The Future of HR Lies in AND – Not in OR

Moving from 'or' to 'and' thinking in HR strategy
06:59 MIN

Moving from 'or' to 'and' thinking in HR strategy

The Future of HR Lies in AND – Not in OR

Understanding global differences in work culture and motivation
06:10 MIN

Understanding global differences in work culture and motivation

The Future of HR Lies in AND – Not in OR

Featured Partners

Related Videos

See all videos
How Machine Learning is turning the Automotive Industry upside down26:55

How Machine Learning is turning the Automotive Industry upside down

Jan Zawadzki

What non-automotive Machine Learning projects can learn from automotive Machine Learning projects48:01

What non-automotive Machine Learning projects can learn from automotive Machine Learning projects

Jan Zawadzki

Finding the unknown unknowns: intelligent data collection for autonomous driving development19:24

Finding the unknown unknowns: intelligent data collection for autonomous driving development

Liang Yu

How computers learn to see – Applying AI to industry22:11

How computers learn to see – Applying AI to industry

Antonia Hahn

Intelligent Data Selection for Continual Learning of AI Functions52:10

Intelligent Data Selection for Continual Learning of AI Functions

Nico Schmidt

Automated Driving - Why is it so hard to introduce31:23

Automated Driving - Why is it so hard to introduce

Sayed Bouzouraa

Developing an AI.SDK38:12

Developing an AI.SDK

Daniel Graff & Andreas Wittmann

Robots 2.0: When artificial intelligence meets steel28:19

Robots 2.0: When artificial intelligence meets steel

Thomas Tomow

Related Articles

View all articles
CH
Chris Heilmann
With AIs wide open - WeAreDevelopers at All Things Open 2025
Last week our VP of Developer Relations, Chris Heilmann, flew to Raleigh, North Carolina to present at All Things Open . An excellent event he had spoken at a few times in the past and this being the “Lucky 13” edition, he didn’t hesitate to come and...
With AIs wide open - WeAreDevelopers at All Things Open 2025
DC
Daniel Cranney
How software is steering vehicle technology
The automotive industry is entering a transformative era, and developers have a unique opportunity to be part of it. Cars are no longer just mechanical machines; they’re sophisticated tech platforms with software at their core. This shift, defined by...
How software is steering vehicle technology

From learning to earning

Jobs that call for the skills explored in this talk.