An overview of the AI training stack, from high-level frameworks like PyTorch down to low-level network fabrics like InfiniBand and RoCE v2.
A demonstration of a basic PyTorch distributed training job on Kubernetes, showing the forward pass, backward pass, and optimizer steps in action.
An explanation of the three primary strategies for distributing model training across many GPUs: data parallel, tensor parallel, and pipeline parallel.
An introduction to the NVIDIA Collective Communications Library (NCCL) and its role in coordinating GPU synchronization through operations like all-reduce.
A demo shows how network contention from multiple jobs increases queue pair latency, which degrades overall training throughput by up to 50%.
A comparison between the slower, general-purpose front-end Ethernet network and the specialized, high-speed back-end network for GPU-to-GPU traffic.
An explanation of key back-end networking technologies, including RDMA for kernel bypass and a comparison of InfiniBand versus RoCE v2 fabrics.
A demonstration of how a transient link failure, or "flap," can cause an entire distributed training job to crash without the ability to self-recover.
An overview of checkpointing as a fault tolerance mechanism to save training progress and recover from failures caused by hardware issues like flapping links or GPU errors.
A summary of critical insights, emphasizing that high-latency queue pairs degrade throughput and that building fault tolerance is essential for reliable AI workloads.
Jobs that call for the skills explored in this talk.