> Markdown version of [/videos/1294-cuda-in-python](https://www.wearedevelopers.com/videos/1294-cuda-in-python). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# CUDA in Python

Tired of choosing between Python productivity and C++ hardware speed? Discover how new CUDA tools finally give developers low-level GPU control for maximum performance directly in Python.

- **Speakers:** Andy Terrel
- **Event:** WeAreDevelopers LIVE
- **Published:** February 5, 2025
- **Duration:** 21:02
- **URL:** https://www.wearedevelopers.com/videos/1294-cuda-in-python

## Summary

NVIDIA's CUDA platform provides high-performance GPU capabilities, but developers have historically faced a trade-off: choose the high productivity of Python frameworks like PyTorch, or achieve maximum hardware speed via low-level C++. The introduction of tools under the CUDA Python umbrella directly bridges this divide. By surfacing low-level device control directly into the Python domain, engineers no longer have to sacrifice performance to build and manage hardware pipelines.

At the forefront of this integration is `nvmath-python`, a library that features auto-tuning and kernel fusion to execute multiple operations—like a matrix multiplication followed by an epilogue bias application—in a single sweep, effectively removing the latency of host-to-device kernel launches. To enhance device-side parallelism, updates to Numba via `numba-cuda` enable warp- and block-level reductions directly in Python using underlying CUB primitives. Compiling these hybrid components is streamlined by the `nvjitlink` tool, which provides just-in-time linking of LLVM IR and PTX representations from both Python and C++ into cohesive, high-speed device modules.

Ultimately, this evolution delivers a native Python device object model that explicitly manages GPU contexts, streams, memory resources, and runtime compilation objects without requiring separate toolchains. By immediately supporting modern architectural features like bfloat16 quantization and generating dynamic bindings for C++ templates, data science and AI teams can write maintainable Python workloads that invoke pre-optimized, native logic with negligible overhead.

**Keywords:** cuda in python, nvmath-python library, numba-cuda compiler, gpu parallelism, kernel fusion optimization, nvjitlink just-in-time linker, python device object model, cublas integration, llvm ir generation, runtime compilation pipeline, ptx virtual machine, hardware acceleration tuning, bfloat16 tensor operations, c++ template bindings

## Chapters

1. **Understanding the CUDA abstraction stack for developers** (00:06) — Different abstraction layers from PyTorch down to PTX expose unique GPU capabilities.
1. **Bridging the productivity and speed gap in Python** (03:02) — CUDA Python eliminates the rigid tradeoff between high-level scripting productivity and low-level hardware performance.
1. **Optimizing performance through kernel fusion with nvMATH** (05:20) — Fusing matrix multiplication and epilogues into a single kernel prevents excessive host-to-device communication overhead.
1. **Enabling device-side function calls with Numba JIT** (08:35) — Just-in-time linking with LLVM grants Python kernels direct internal access to compiled C++ device functions.
1. **Controlling parallelism and block reductions in Python** (12:18) — CUDA Cooperative enables granular control over threaded hardware blocks without exiting the Python execution environment.
1. **Exposing low-level hardware features via Numba CUDA** (14:55) — Translating C++ templates allows Python developers immediate implementation of optimized hardware floating point types.
1. **Managing host-side GPU resources using Python objects** (17:32) — A native object model replaces complex bindings for instantiating streams, events, and programmatic compiler representations.
1. **Accessing educational resources for Python accelerator computing** (20:13) — Forking the GitHub repository and evaluating reference examples immediately builds proficiency in accelerated Python development.

## Related Moments

- [The expanded CUDA ecosystem and native Python support](https://www.wearedevelopers.com/videos/100221-cuda-python-gpu-programming-for-the-modern-developer) (from "CUDA Python: GPU programming for the modern developer")
- [Simplifying parallel programming with the CUDA ecosystem](https://www.wearedevelopers.com/videos/859-accelerating-python-on-gpus) (from "Accelerating Python on GPUs")
- [Accelerating math operations and kernel fusion with nvmath-python](https://www.wearedevelopers.com/videos/1521-accelerating-python-on-gpus) (from "Accelerating Python on GPUs")
- [Exploring the CUDA ecosystem and levels of abstraction](https://www.wearedevelopers.com/videos/1521-accelerating-python-on-gpus) (from "Accelerating Python on GPUs")
- [Accelerating script execution with CuPy and Numba kernels](https://www.wearedevelopers.com/videos/100221-cuda-python-gpu-programming-for-the-modern-developer) (from "CUDA Python: GPU programming for the modern developer")
- [Accelerating compute with focused developer tools](https://www.wearedevelopers.com/videos/100070-from-ai-assistance-to-agentic-systems-scaling-sovereign-ai-in-banking) (from "From AI Assistance to Agentic Systems: Scaling Sovereign AI in Banking")

## Related Articles

- [What’s the latest in NVIDIA CUDA Python](https://www.wearedevelopers.com/magazine/568-what-s-the-latest-in-nvidia-cuda-python)
- [Dev Digest 157: CUDA in Python, Gemini Code Assist and Back-dooring LLMs](https://www.wearedevelopers.com/magazine/557-dev-digest-157-cuda-in-python-gemini-code-assist-and-back-dooring-llms)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [How we Build The Software of Tomorrow](https://www.wearedevelopers.com/magazine/120-how-we-build-the-software-of-tomorrow)

## Related Jobs

- [Hardware-naher Algorithmenentwickler](https://www.wearedevelopers.com/jobs/ext/1684535-hardware-naher-algorithmenentwickler) at **ZEISS Group**
- [Principal Software Engineer](https://www.wearedevelopers.com/jobs/ext/146806-principal-software-engineer) at **Twilio**
- [Software Engineer L3](https://www.wearedevelopers.com/jobs/ext/1840587-software-engineer-l3) at **Twilio**
- [Software Engineer](https://www.wearedevelopers.com/jobs/ext/1304791-software-engineer) at **Bitpanda**
- [Principal Software Engineer](https://www.wearedevelopers.com/jobs/ext/267587-principal-software-engineer) at **Twilio**
- [Principal Software Engineer](https://www.wearedevelopers.com/jobs/ext/115163-principal-software-engineer) at **Twilio**