> Markdown version of [/events/world-congress-2024/sessions/445-pytorch-train-loop](https://www.wearedevelopers.com/events/world-congress-2024/sessions/445-pytorch-train-loop). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# PyTorch train loop from scratch - from local to distributed


- **Date:** Friday, Jul 19, 2024
- **Time:** 16:20–16:50 (30 min)
- **Room:** STAGE 3 (120)
- **Event:** World Congress 2024

## Description

Dive into PyTorch internals by live coding a training loop from scratch. We'll start simple then level up, going from a basic local loop to a scalable distributed pipeline. See how the sausage is made as we build blocks like optimization, serialization, checkpointing, and broadcasting. Monitor model convergence graphs in real-time as we train a transformer language model on custom datasets. By the finale of this hands-on coding session, you'll have the skills to launch robust distributed PyTorch jobs on the cloud, opening doors to cutting-edge AI and impactful model building.

## Speaker

### [Dennis Bappert](https://www.wearedevelopers.com/@dennis-bappert)

Sr. Solutions Architect


## Related talks at this congress

- [Supercharge your cloud-native applications with Generative AI](https://www.wearedevelopers.com/events/world-congress-2024/sessions/53-supercharge-your) — Cedric Clyburn
- [Unlocking the Power of AI: Accessible Language Model Tuning for All](https://www.wearedevelopers.com/events/world-congress-2024/sessions/106-unlocking-the-power) — Cedric Clyburn, @technicallylegare
- [Make it simple, using generative AI to accelerate learning](https://www.wearedevelopers.com/events/world-congress-2024/sessions/276-make-it-simple-using) — Duan Lightfoot
- [Remote Mob Programming - At home, but not alone](https://www.wearedevelopers.com/events/world-congress-2024/sessions/47-remote-mob) — Joshua Töpfer