Running AI Workloads in Containers and Kubernetes

Kubernetes is quickly becoming the platform of choice for running machine learning and AI workloads in the cloud. However, running these workloads efficiently poses unique challenges, from resource management to performance optimization.

In this talk, we dive into the details of how GPUs are made available to such workloads when running under Kubernetes. As part of this, we discuss various options for sharing GPUs between them. These techniques include simple time-slicing, MPS, and MIG.

By the end of this session, attendees will have a comprehensive understanding of how GPU support in Kubernetes works under the hood, as well as the knowledge required to make the most efficient use of GPUs in their own applications. We conclude with a demo.