> Markdown version of [/events/world-congress-2024/sessions/204-running-ai-workloads](https://www.wearedevelopers.com/events/world-congress-2024/sessions/204-running-ai-workloads). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Running AI Workloads in Containers and Kubernetes


- **Date:** Thursday, Jul 18, 2024
- **Time:** 16:10–16:40 (30 min)
- **Room:** STAGE 8 (120)
- **Event:** World Congress 2024

## Description

Kubernetes is quickly becoming the platform of choice for running machine learning and AI workloads in the cloud. However, running these workloads efficiently poses unique challenges, from resource management to performance optimization. 

In this talk, we dive into the details of how GPUs are made available to such workloads when running under Kubernetes. As part of this, we discuss various options for sharing GPUs between them. These techniques include simple time-slicing, MPS, and MIG.

By the end of this session, attendees will have a comprehensive understanding of how GPU support in Kubernetes works under the hood, as well as the knowledge required to make the most efficient use of GPUs in their own applications. We conclude with a demo.

## Speakers

### [Evan Lezar](https://www.wearedevelopers.com/@evan-lezar)

Senior Systems Software Engineer, NVIDIA
### [Kevin Klues](https://www.wearedevelopers.com/@kevin-klues)

Distinguished Engineer at NVIDIA


## Related talks at this congress

- [From foundation model to hosted AI solution in minutes](https://www.wearedevelopers.com/events/world-congress-2024/sessions/183-from-foundation) — Kevin Klues, Stephan Schosser
- [Containers and Kubernetes made easy: Deep dive into Podman Desktop and new AI capabilities](https://www.wearedevelopers.com/events/world-congress-2024/sessions/87-containers-and) — Stevan Le Meur
- [Supercharge your cloud-native applications with Generative AI](https://www.wearedevelopers.com/events/world-congress-2024/sessions/53-supercharge-your) — Cedric Clyburn
- [Using Containers to deploy AI Models across our microscopy platform](https://www.wearedevelopers.com/events/world-congress-2024/sessions/312-using-containers-to) — Sebastian Rhode