WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Feb 23, 2024

Accelerating Python on GPUs

Paul Graham

Paul Graham proves you can unlock massive GPU compute power in Python without touching C++. Discover the multi-tiered path from zero-code pandas acceleration to custom PyCUDA integrations.

Pause
Mute Enter Fullscreen
#1 about 3 min

History and scale of NVIDIA GPU computing

The transition of GPUs from graphics rendering engines to general-purpose computing processors and deep learning accelerators.

#2 about 4 min

Comparing single-threaded CPU performance and parallel GPU architectures

How physical limitations in single-core CPU scaling drove the adoption of parallel GPU processing architectures.

#3 about 2 min

Offloading computational bottlenecks from CPUs to GPUs

The fundamental workflow of delegating time-consuming parallel loops to the GPU while the CPU handles sequential logic.

#4 about 6 min

Analyzing hardware components inside a modern GPU architecture

How streaming multiprocessors and specialized cores use high-thread-count execution to hide latency dynamically.

#5 about 2 min

Scaling applications across multi-node GPU clusters and architectures

How data movement considerations and hardware technologies shape multi-gpu communication and system scale-out operations.

#6 about 5 min

Simplifying parallel programming with the CUDA ecosystem

The evolution of abstraction layers and compilers that allow developers to execute C++ and Python code on GPUs without graphics APIs.

#7 about 4 min

Leveraging tensor cores for deep neural network execution

Exploiting specialized matrix multiply-accumulate hardware through automated mixed-precision libraries and lower-level developer interfaces.

#8 about 3 min

Exploring high-level Python frameworks for accelerated enterprise artificial intelligence

A review of application-specific libraries for routing, speech recognition, large language models, and physics-informed neural networks.

#9 about 4 min

Refactoring data science workflows using Rapids QDF and Pandas

Achieving extreme performance scaling in data preparation pipelines by swapping standard libraries for GPU-optimized equivalents.

#10 about 4 min

Transitioning into custom parallel computing implementations using Python

A stepwise approach to optimizing unique workloads by introducing specific computational libraries and incremental compilation directives.

#11 about 2 min

Extending distributed parallel processing capabilities using QNumeric arrays

Distributing implicit matrix parallelization techniques across thousands of nodes transparently through a drop-in NumPy replacement.

#12 about 4 min

Applying targeted just-in-time compilation rules with Python Numba

Controlling low-level memory allocation and individual thread execution paths using typed decorators within analytical Python logic.

#13 about 2 min

Implementing specialized device kernels using CuPy and PyCUDA

Writing pure C or C++ kernel structures to exact maximal control and performance within Python scripting environments.

#14 about 3 min

Analyzing logic and performance metrics via developer profiling tools

Isolating execution bottlenecks, race conditions, and out-of-bounds memory leaks using correctness checkers and roofline analysis utilities.

#15 about 5 min

Accessing preconfigured environments via the NGC software repository

Obtaining pre-trained state-of-the-art models and monthly optimized containers from a centralized machine learning distribution registry.

#16 about 4 min

Deploying inference architectures on mobile edge computing endpoints

Adapting data center memory boundaries and network architectures to function securely alongside strict low-power specifications.

#17 about 2 min

Navigating emerging hardware standardization in vendor programming ecosystems

The migration from proprietary optimization languages toward universal backend compilers that support distinct hardware architectures.

#18 about 2 min

Evaluating absolute performance thresholds between Python and procedural syntaxes

Understanding when high-level scripting wrappers produce overhead compared to native implementation code during aggressive optimization cycles.

#19 about 3 min

Shaping future physical logic gates with deep learning requirements

How algorithmic phenomena like transformer models directly inspire novel transistor arrangements and mixed-precision capabilities.

#20 about 3 min

Evaluating remote hardware allocations for localized infrastructure constraints

Using local subsystems or cloud-hosted instances to experiment with GPU-accelerated code when strict budget boundaries exist.

#21 about 2 min

Forecasting compiler intelligence and declarative syntactic features in computing

Anticipating runtime automation protocols that abstract complex data pipeline topologies from standard algorithmic structures.

#22 about 2 min

Container orchestration and scalability dynamics affecting cloud infrastructure performance

Developing enclosed analytical experiments via localized instances before deploying complex training epochs across large remote computational clusters.

Matching moments

3:32 min

Evolution of general purpose GPU computing and Python

Paul Graham Paul Graham · World Congress 2025

1:30 min

Exploring the Python-enabled GPU programming software stack

Paul Graham Paul Graham · World Congress 2026 Europe

2:15 min

Introduction to CUDA and general-purpose GPU computing

Paul Graham Paul Graham · World Congress 2026 Europe

1:49 min

Accelerating script execution with CuPy and Numba kernels

Paul Graham Paul Graham · World Congress 2026 Europe

1:37 min

Accelerating compute with focused developer tools

Julia Koch Julia Koch +1 · World Congress 2026 Europe

2:27 min

Exploring the CUDA ecosystem and levels of abstraction

Paul Graham Paul Graham · World Congress 2025

More from WeAreDevelopers LIVE

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude thumbnail
6:16
4 days ago

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude

Chris Heilmann, Daniel Cranney

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas thumbnail
5:46
9 days ago

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas

Chris Heilmann, Daniel Cranney, Marc Plogas

WeAreDevelopers - Of Agents and Robots thumbnail
1:04:42
9 days ago

WeAreDevelopers - Of Agents and Robots

Chris Heilmann, Daniel Cranney, Marc Plogas

WeAreDevelopers LIVE - How DevRel Makes Tech More Human thumbnail
48:06
16 days ago

WeAreDevelopers LIVE - How DevRel Makes Tech More Human

Chris Heilmann, Daniel Cranney

Fake or News: Smart Glasses in Court, Medical Advice Watches & Mushroom Electonics - Jason Torres thumbnail
7:41
17 days ago

Fake or News: Smart Glasses in Court, Medical Advice Watches & Mushroom Electonics - Jason Torres

Chris Heilmann, Daniel Cranney

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
30 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 25, 2026 · 11:00–11:30

Stage 5

Managing GPUs by Just Asking, Infrastructure in the Age of MCP

Jessica Garson Beauchemin

Developer Relations Lead, Community at Runpod

Jessica Garson Beauchemin
Open session

World Congress 2026 North America

September 24, 2026 · 14:10–14:40

Stage 6

Compute for your AI model: GPUs, LPUs, TPUs and beyond..

Kushaagra Goyal

Tech Lead at Rubrik, ex-CTO at Gan.AI, ex-Databricks

Kushaagra Goyal
Open session

World Congress 2026 North America

September 25, 2026 · 15:30–16:00

Stage 7

Trust, But Verify: Continuous GPU Validation at Scale

Kyle Bell

VP of AI @ TensorWave

Kyle Bell
Open session

World Congress 2026 North America

September 23, 2026 · 10:45–12:45

Stage 10

Agents That Own Their Inference: Building Production AI Agents on Dedicated GPUs

Duan Lightfoot

Sr. AI Engineer, Akamai

Duan Lightfoot
Open session

World Congress 2026 North America

September 24, 2026 · 14:10–14:40

Stage 1

Anatomy of an AI Request: Where Latency and Cost Are Really Born

Dan Fu

VP of Kernels at Together AI

Dan Fu
Open session

World Congress 2026 North America

September 24, 2026 · 11:00–11:30

Stage 1

Application-Defined Compute: Rethinking Infrastructure for AI Applications

Anurag Goel

Founder and CEO of Render

Anurag Goel

Related videos

Accelerating Python on GPUs thumbnail
22:18
World Congress 2024 • about 2 years ago

Accelerating Python on GPUs

Paul Graham

Accelerating Python on GPUs thumbnail
24:39
World Congress 2025 • about 1 year ago

Accelerating Python on GPUs

Paul Graham

CUDA Python: GPU programming for the modern developer thumbnail
29:43
World Congress 2026 Europe • about 2 months ago

CUDA Python: GPU programming for the modern developer

Paul Graham

Coffee with Developers - Stephen Jones - NVIDIA thumbnail
37:32
Coffee With Developers • over 1 year ago

Coffee with Developers - Stephen Jones - NVIDIA

Stephen Jones

CUDA in Python thumbnail
21:02
LIVE • over 1 year ago

CUDA in Python

Andy Terrel

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA thumbnail
22:07
World Congress 2024 • about 2 years ago

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA

Ankit Patel

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.