Anshul Jindal & Martin Piercy
Your Next AI Needs 10,000 GPUs. Now What?
#1about 2 minutes
Introduction to large-scale AI infrastructure challenges
An overview of the topics to be covered, from the progress of generative AI to the compute requirements for training and inference.
#2about 4 minutes
Understanding the fundamental shift to generative AI
Generative AI creates novel content, moving beyond prediction to unlock new use cases in coding, content creation, and customer experience.
#3about 6 minutes
Using NVIDIA NIMs and blueprints to deploy models
NVIDIA Inference Microservices (NIMs) and blueprints provide pre-packaged, optimized containers to quickly deploy models for tasks like retrieval-augmented generation (RAG).
#4about 4 minutes
An overview of the AI model development lifecycle
Building a production-ready model involves a multi-stage process including data curation, distributed training, alignment, optimized inference, and implementing guardrails.
#5about 6 minutes
Understanding parallelism techniques for distributed AI training
Training massive models requires splitting them across thousands of GPUs using tensor, pipeline, and data parallelism to manage compute and communication.
#6about 2 minutes
The scale of GPU compute for training and inference
Training large models like Llama requires millions of GPU hours, while inference for a single large model can demand a full multi-GPU server.
#7about 3 minutes
Key hardware and network design for AI infrastructure
Effective multi-node training depends on high-speed interconnects like NVLink and network architectures designed to minimize communication latency between GPUs.
#8about 3 minutes
Accessing global GPU capacity with DGX Cloud Lepton
NVIDIA's DGX Cloud Lepton is a marketplace connecting developers to a global network of cloud partners for scalable, on-demand GPU compute.
Related jobs
Jobs that call for the skills explored in this talk.
Wilken GmbH
Ulm, Germany
Senior
Kubernetes
AI Frameworks
+3
Picnic Technologies B.V.
Amsterdam, Netherlands
Intermediate
Senior
Python
Structured Query Language (SQL)
+1
Matching moments
04:57 MIN
Increasing the value of talk recordings post-event
Cat Herding with Lions and Tigers - Christian Heilmann
01:32 MIN
Organizing a developer conference for 15,000 attendees
Cat Herding with Lions and Tigers - Christian Heilmann
02:54 MIN
Automating video post-production with local scripts
Cat Herding with Lions and Tigers - Christian Heilmann
02:39 MIN
Establishing a single source of truth for all data
Cat Herding with Lions and Tigers - Christian Heilmann
03:28 MIN
Why corporate AI adoption lags behind the hype
What 2025 Taught Us: A Year-End Special with Hung Lee
04:27 MIN
Moving beyond headcount to solve business problems
What 2025 Taught Us: A Year-End Special with Hung Lee
03:15 MIN
The future of recruiting beyond talent acquisition
What 2025 Taught Us: A Year-End Special with Hung Lee
03:39 MIN
Breaking down silos between HR, tech, and business
What 2025 Taught Us: A Year-End Special with Hung Lee
Featured Partners
Related Videos
22:07WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA
Ankit Patel
32:19A Deep Dive on How To Leverage the NVIDIA GB200 for Ultra-Fast Training and Inference on Kubernetes
Kevin Klues
23:01Efficient deployment and inference of GPU-accelerated LLMs
Adolf Hohl
31:25Unveiling the Magic: Scaling Large Language Models to Serve Millions
Patrick Koss
28:38Exploring LLMs across clouds
Tomislav Tipurić
35:16How AI Models Get Smarter
Ankit Patel
24:26AI Factories at Scale
Thomas Schmidt
32:27Generative AI power on the web: making web apps smarter with WebGPU and WebNN
Christian Liebel
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
Forschungszentrum Jülich GmbH
Jülich, Germany
Intermediate
Senior
Linux
Docker
AI Frameworks
Machine Learning
NVIDIA
Canton de Plaisir, France
Senior
C++
Python
PyTorch
NVIDIA Corporation
Remote
Senior
C++
DevOps
Python
Docker
+1
NVIDIA
Municipality of Madrid, Spain
Senior
C++
DevOps
Python
Docker
Kubernetes
UL Solutions
Barcelona, Spain
Python
Machine Learning
Nvidia
Bramley, United Kingdom
Senior
DevOps
Python
OpenCL
Gitlab
Docker
+2