How Robots Learn to be Robots

How do you safely teach a robot to handle dangerous tasks? By letting it fail a million times in a digital world first.

#1about 3 minutes

The evolution from digital agents to physical AI

Physical AI extends digital agentic systems into the physical world, enabling robots to understand and act based on sensory input and instructions.

#2about 4 minutes

The three-computer solution for robotics challenges

The high cost of data and physical testing is solved by a three-computer workflow using simulation for data generation, a separate system for training, and simulation again for validation.

#3about 3 minutes

Understanding reinforcement and imitation learning for robots

Robots learn skills through reinforcement learning by trial and error with rewards, or through imitation learning by mimicking expert demonstrations.

#4about 2 minutes

Building digital twins with Omniverse and Isaac Sim

NVIDIA Omniverse is a development platform based on OpenUSD for creating digital twins, with Isaac Sim providing a dedicated application for robot simulation.

#5about 2 minutes

Scaling training data with simulated teleoperation

A small number of human demonstrations collected via teleoperation in Isaac Sim can be algorithmically scaled into a large, diverse, and photorealistic synthetic dataset.

#6about 3 minutes

Generating photorealistic data with Cosmos foundation models

Cosmos is a platform of world foundation models, including Cosmos Transfer, which uses control nets to transform basic simulation outputs into photorealistic videos.

#7about 3 minutes

Generating novel robot scenarios with Isaac GR00T Dreams

The Isaac GR00T Dreams workflow uses a post-trained Cosmos Predict model to generate new robot behaviors from a single image, which are then filtered and labeled to create training data.

#8about 2 minutes

A hybrid data strategy for robust foundation models

Robust robot foundation models like Isaac GR00T are trained by combining vast internet video data, high-quality human demonstrations, and large-scale synthetic data.

#9about 1 minute

Testing and deploying robots in large-scale simulations

Before real-world deployment, trained robot models can be rigorously tested at scale within digital twins of complex environments like factories using OpenUSD.