WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2021 • Jun 30, 2021

On the straight and narrow path - How to get cars to drive themselves using reinforcement learning and trajectory optimization

Francis Powlesland , Elena Kotljarova

Can a car master an unknown track without pre-programmed physics? Discover how Q-learning and trajectory optimization teach autonomous vehicles to drive themselves from scratch.

Pause
Mute Enter Fullscreen
#1 about 5 min

Limitations of pre-trained models in autonomous driving

Reinforcement learning offers an alternative to static pre-trained models for self-driving vehicles by dynamically navigating unknown environments.

#2 about 3 min

Augmented intelligence for customized driving experiences

Artificial intelligence can advise drivers and adapt to personal styles without taking full autonomous control of the vehicle.

#3 about 3 min

Establishing a human driving performance lap baseline

Setting a manual lap time benchmark provides a physical optimization target for the artificial intelligence model.

#4 about 2 min

Initial artificial intelligence exploration and strategy experimentation

Early training laps exhibit erratic vehicle behavior as the unweighted algorithm experiments with new driving strategies across track segments.

#5 about 5 min

Evaluating model accuracy and diminishing training returns

Extending training periods exponentially yields progressively smaller lap time improvements for autonomous racing agents.

#6 about 6 min

Conceptual foundations of reinforcement learning agents

Autonomous agents maximize rewards by repeatedly testing defined actions and discovering consequences within entirely unmapped environments.

#7 about 6 min

Implementing Q-learning formulas for trajectory optimization

Mapping discrete track states to specific decisions via Q-tables creates a structured memory base for real-time trajectory updates.

#8 about 3 min

Tuning hyperparameters for reinforcement learning agent outcomes

Adjusting the learning rate, discount factor, and epsilon values balances immediate reward gratification with finding the optimal long-term strategy.

#9 about 2 min

System architecture for the physical racing demonstration

The physical demonstration stack combines hardware microcontrollers, telemetry message brokers, and web frameworks to execute the continuous evaluation loop.

#10 about 5 min

Transitioning self-learning vehicles to real-world environments

Applying tracking algorithms to physical vehicle dynamics enables active suspension and steering adjustments upon actual consumer roads.

#11 about 7 min

Mitigating local minimums and managing data quality

Ensuring high-quality training inputs and properly defined optimization boundaries prevents autonomous models from stalling inside sub-optimal algorithmic states.

Matching moments

4:55 min

Audience Q&A on autonomous driving models and data

Liang Yu · World Congress 2022

2:52 min

Introduction to safety-critical machine learning in automotive contexts

Jan Zawadzki · World Congress 2022

3:55 min

Technical catalysts driving real-world artificial intelligence

Clemens Wasner Clemens Wasner +3 · World Congress 2026 Europe

3:05 min

Introduction to the speakers and topic

Ulrich Wurstbauer +1 · LIVE

1:43 min

Optimizing AI model execution for in-car inference

Daniel Graff +1 · World Congress 2021

3:37 min

Transitioning automated driving to neural networks and model-based perception

Katrin Lehmann Katrin Lehmann +1 · World Congress 2025

More from World Congress 2021

Micro Frontends with Module Federation: Why and How? thumbnail
45:40
about 5 years ago

Micro Frontends with Module Federation: Why and How?

Manfred Steyer

Data Science on Software Data thumbnail
48:10
about 5 years ago

Data Science on Software Data

Markus Harrer

The journey from developer to devops - what i've learnt along the way thumbnail
42:44
about 5 years ago

The journey from developer to devops - what i've learnt along the way

Liam Hurrel, Alireza Chegini

How Data is Shaping our Games thumbnail
43:30
about 5 years ago

How Data is Shaping our Games

Johanna Pirker

Cross platform Augmented Reality development with React Native thumbnail
40:50
about 5 years ago

Cross platform Augmented Reality development with React Native

Vladimir Novick

How to Build a Monetization Strategy Based on User's in-app Behavior thumbnail
38:59
about 5 years ago

How to Build a Monetization Strategy Based on User's in-app Behavior

Natalie Sieradzki

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 25, 2026 · 11:40–12:10

Stage 9

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan
Open session

World Congress 2026 North America

September 24, 2026 · 16:10–16:40

Mainstage

The future of workforce, at high speed

Daniela Dimitrova

Vice President IT, CIO at Mercedes-Benz North America

Daniela Dimitrova
Open session

World Congress 2026 North America

September 24, 2026 · 17:30–18:00

Stage 6

No Single Model to Rule Them All: Building Resilient AI Agents Across Open & Closed LLMs

Emmanuel Acheampong

Senior Manager Developer Relations at Crusoe AI

Emmanuel Acheampong
Open session

World Congress 2026 North America

September 23, 2026 · 15:40–16:10

Stage 2

Lean Intelligence: Lessons from GitHub Copilot Data Science Efforts

Rahul Pandita

Researcher and Technical Advisor at Microsoft

Rahul Pandita
Open session

World Congress 2026 North America

September 24, 2026 · 14:10–14:40

Stage 5

Edge AI: Running Agentic Intelligence Where Internet Can't Reach

Nitin Eusebius

AWS - Principal Solutions Architect

Nitin Eusebius
Open session

World Congress 2026 North America

September 24, 2026 · 11:40–12:10

Stage 4

Taming Rogue Agents: Observability-Driven Evaluation for Production Reliability

Anagha Rumade, Anjana Umapathy, Apoorva Jaiswal

Anagha Rumade
Anjana Umapathy
Apoorva Jaiswal

Related videos

How Machine Learning is turning the Automotive Industry upside down thumbnail
26:55
LIVE • almost 6 years ago

How Machine Learning is turning the Automotive Industry upside down

Jan Zawadzki

Getting Started with Machine Learning thumbnail
44:11
LIVE • almost 5 years ago

Getting Started with Machine Learning

Alexandra Waldherr

What non-automotive Machine Learning projects can learn from automotive Machine Learning projects thumbnail
48:01
World Congress 2022 • about 4 years ago

What non-automotive Machine Learning projects can learn from automotive Machine Learning projects

Jan Zawadzki

Intelligent Data Selection for Continual Learning of AI Functions thumbnail
52:10
LIVE • over 4 years ago

Intelligent Data Selection for Continual Learning of AI Functions

Nico Schmidt

How to develop an autonomous car end-to-end: Robotic Drive and the mobility revolution thumbnail
1:18:27
LIVE • about 6 years ago

How to develop an autonomous car end-to-end: Robotic Drive and the mobility revolution

Ulrich Wurstbauer, Mohamed Nassar

Fine-Tuning Small Language Models for Agentic AI thumbnail
29:18
World Congress 2026 Europe • 2 months ago

Fine-Tuning Small Language Models for Agentic AI

Björn Buchhold

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.