WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2025 • Aug 20, 2025 • Session details

A Deep Dive on How To Leverage the NVIDIA GB200 for Ultra-Fast Training and Inference on Kubernetes

Kevin Klues

The NVIDIA GB200 joins 72 distributed GPUs into one supercomputer. Learn how the Compute Domain abstraction and Kubernetes DRA instantly solve the massive multi-node networking nightmare.

Pause
Mute Enter Fullscreen
#1 about 1 min

Supporting NVIDIA GB200 GPUs on Kubernetes

Adapting Kubernetes to support the unique architecture of the new GB200 GPU family.

#2 about 2 min

Scaling beyond single-node GPU communication limits

How multi-node NVLink uses NV switches to connect GPUs across separate physical machines.

#3 about 2 min

Kubernetes GPU operator limitations with GB200

Why out-of-the-box cluster configurations fail to provide secure shared memory access across multi-node setups.

#4 about 1 min

Enabling secure memory exchange with IMEX

How IMEX allows isolated GPU processes to directly read and write remote memory over NVLink.

#5 about 2 min

CUDA APIs for IMEX fabric attached memory

Mapping remote hardware through local pointers using specific CUDA memory handle exports.

#6 about 6 min

Four partitioning layers for IMEX access

Managing physical wiring, switch partitions, dynamic soft domains, and per-process channel permissions.

#7 about 5 min

Orchestrating dynamic IMEX domains for workloads

Configuring daemons to build secure point-to-point topologies mirroring Kubernetes pod deployments.

#8 about 5 min

Simplifying cluster setups using compute domains

Hiding multi-node communication wiring beneath custom API server objects and resource claim templates.

#9 about 3 min

Automating compute domains via the DRA driver

How controllers and kubelet plugins coordinate IMEX daemons and inject channel permissions sequentially.

#10 about 2 min

Routing cross-rack traffic seamlessly with NCCL

Binding segmented NVLink partitions automatically so higher-level libraries fallback to InfiniBand between isolated domains.

#11 about 7 min

Demonstrating multi-node high bandwidth memory transfers

Measuring memory read and write speeds by running an MPI workload across clustered compute resources.

#12 about 2 min

Cluster prerequisites to deploy compute domains

Enabling the required feature flags, CDIs, and custom NVIDIA daemons within your infrastructure.

Matching moments

1:37 min

Unlocking direct GPU access within managed Kubernetes platforms

Kevin Klues Kevin Klues

2:43 min

Bursting GPU capacity over hybrid Kubernetes networks

Jeremy Murray Jeremy Murray · World Congress 2026 Europe

2:59 min

Scaling up and scaling out GPU clusters

Michael Kagan Michael Kagan +1 · World Congress 2026 Europe

3:10 min

Scaling customized inference models with NVIDIA NIM

Anshul Jindal Anshul Jindal · World Congress 2025

1:54 min

Scaling applications across multi-node GPU clusters and architectures

Paul Graham Paul Graham · LIVE

1:20 min

How the Kubernetes AI scheduler enables fractional GPU sharing

Piotr Zaniewski Piotr Zaniewski · World Congress 2026 Europe

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail
24:33
11 months ago

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing

Behind the Code: How Women Are Powering the Future of AI thumbnail
30:28
about 1 year ago

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel, Laura Moritz, Madalina Florean

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail
28:12
about 1 year ago

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe, Konstantin Vinogradov, Nils Eiteneyer, Stephan Wirris

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail
28:19
about 1 year ago

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler, Jasmin Kaiser, Jeff Hausmann, Sonja Alvarez

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail
28:09
about 1 year ago

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher, Jan Oberhauser, Keno Dreßel, Mirko Novakovic

AI Sovereignty: What Does It Take? thumbnail
29:13
about 1 year ago

AI Sovereignty: What Does It Take?

Boris Hecker, Clemens Wasner, Robin Hermann, Sonja Alvarez

Related videos

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬 thumbnail
24:23
World Congress 2026 Europe • 3 months ago

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬

Jeremy Murray

Instant KAI Sandboxes with vCluster: Multi-Tenant, Multi-Scheduler GPU Sharing thumbnail
27:54
World Congress 2026 Europe • 3 months ago

Instant KAI Sandboxes with vCluster: Multi-Tenant, Multi-Scheduler GPU Sharing

Piotr Zaniewski

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA thumbnail
22:07
World Congress 2024 • about 2 years ago

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA

Ankit Patel

The Gashlycrumb Tinies of AI Networking You Must Know (or Languish!) thumbnail
28:19
Europe 2026 Virtual • 3 months ago

The Gashlycrumb Tinies of AI Networking You Must Know (or Languish!)

Lerna Ekmekcioglu

Your Next AI Needs 10,000 GPUs. Now What? thumbnail
29:52
World Congress 2025 • about 1 year ago

Your Next AI Needs 10,000 GPUs. Now What?

Anshul Jindal, Martin Piercy

Accelerating Python on GPUs thumbnail
59:43
LIVE • over 2 years ago

Accelerating Python on GPUs

Paul Graham

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.