WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE β€’ Feb 23, 2024

Accelerating Python on GPUs

Paul Graham

Paul Graham proves you can unlock massive GPU compute power in Python without touching C++. Discover the multi-tiered path from zero-code pandas acceleration to custom PyCUDA integrations.

Pause
Mute Enter Fullscreen
#1 about 3 min

History and scale of NVIDIA GPU computing

The transition of GPUs from graphics rendering engines to general-purpose computing processors and deep learning accelerators.

#2 about 4 min

Comparing single-threaded CPU performance and parallel GPU architectures

How physical limitations in single-core CPU scaling drove the adoption of parallel GPU processing architectures.

#3 about 2 min

Offloading computational bottlenecks from CPUs to GPUs

The fundamental workflow of delegating time-consuming parallel loops to the GPU while the CPU handles sequential logic.

#4 about 6 min

Analyzing hardware components inside a modern GPU architecture

How streaming multiprocessors and specialized cores use high-thread-count execution to hide latency dynamically.

#5 about 2 min

Scaling applications across multi-node GPU clusters and architectures

How data movement considerations and hardware technologies shape multi-gpu communication and system scale-out operations.

#6 about 5 min

Simplifying parallel programming with the CUDA ecosystem

The evolution of abstraction layers and compilers that allow developers to execute C++ and Python code on GPUs without graphics APIs.

#7 about 4 min

Leveraging tensor cores for deep neural network execution

Exploiting specialized matrix multiply-accumulate hardware through automated mixed-precision libraries and lower-level developer interfaces.

#8 about 3 min

Exploring high-level Python frameworks for accelerated enterprise artificial intelligence

A review of application-specific libraries for routing, speech recognition, large language models, and physics-informed neural networks.

#9 about 4 min

Refactoring data science workflows using Rapids QDF and Pandas

Achieving extreme performance scaling in data preparation pipelines by swapping standard libraries for GPU-optimized equivalents.

#10 about 4 min

Transitioning into custom parallel computing implementations using Python

A stepwise approach to optimizing unique workloads by introducing specific computational libraries and incremental compilation directives.

#11 about 2 min

Extending distributed parallel processing capabilities using QNumeric arrays

Distributing implicit matrix parallelization techniques across thousands of nodes transparently through a drop-in NumPy replacement.

#12 about 4 min

Applying targeted just-in-time compilation rules with Python Numba

Controlling low-level memory allocation and individual thread execution paths using typed decorators within analytical Python logic.

#13 about 2 min

Implementing specialized device kernels using CuPy and PyCUDA

Writing pure C or C++ kernel structures to exact maximal control and performance within Python scripting environments.

#14 about 3 min

Analyzing logic and performance metrics via developer profiling tools

Isolating execution bottlenecks, race conditions, and out-of-bounds memory leaks using correctness checkers and roofline analysis utilities.

#15 about 5 min

Accessing preconfigured environments via the NGC software repository

Obtaining pre-trained state-of-the-art models and monthly optimized containers from a centralized machine learning distribution registry.

#16 about 4 min

Deploying inference architectures on mobile edge computing endpoints

Adapting data center memory boundaries and network architectures to function securely alongside strict low-power specifications.

#17 about 2 min

Navigating emerging hardware standardization in vendor programming ecosystems

The migration from proprietary optimization languages toward universal backend compilers that support distinct hardware architectures.

#18 about 2 min

Evaluating absolute performance thresholds between Python and procedural syntaxes

Understanding when high-level scripting wrappers produce overhead compared to native implementation code during aggressive optimization cycles.

#19 about 3 min

Shaping future physical logic gates with deep learning requirements

How algorithmic phenomena like transformer models directly inspire novel transistor arrangements and mixed-precision capabilities.

#20 about 3 min

Evaluating remote hardware allocations for localized infrastructure constraints

Using local subsystems or cloud-hosted instances to experiment with GPU-accelerated code when strict budget boundaries exist.

#21 about 2 min

Forecasting compiler intelligence and declarative syntactic features in computing

Anticipating runtime automation protocols that abstract complex data pipeline topologies from standard algorithmic structures.

#22 about 2 min

Container orchestration and scalability dynamics affecting cloud infrastructure performance

Developing enclosed analytical experiments via localized instances before deploying complex training epochs across large remote computational clusters.

Matching moments

3:32 min

Evolution of general purpose GPU computing and Python

Paul Graham Paul Graham Β· WWC 2025

1:30 min

Exploring the Python-enabled GPU programming software stack

Paul Graham Paul Graham Β· WWC Europe 2026

2:15 min

Introduction to CUDA and general-purpose GPU computing

Paul Graham Paul Graham Β· WWC Europe 2026

1:49 min

Accelerating script execution with CuPy and Numba kernels

Paul Graham Paul Graham Β· WWC Europe 2026

1:37 min

Accelerating compute with focused developer tools

Julia Koch Julia Koch +1 Β· WWC Europe 2026

2:27 min

Exploring the CUDA ecosystem and levels of abstraction

Paul Graham Paul Graham Β· WWC 2025

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
5 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
5 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
12 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
19 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
26 days ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail
5:55
about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Compute for your AI model: GPUs, LPUs, TPUs and beyond..

Kushaagra Goyal

Tech Lead at Rubrik, ex-CTO at Gan.AI, ex-Databricks

Kushaagra Goyal
Open session

World Congress 2026 North America

Trust, But Verify: Continuous GPU Validation at Scale

Kyle Bell

VP of AI @ TensorWave

Kyle Bell
Open session

World Congress 2026 North America

Agents That Own Their Inference: Building Production AI Agents on Dedicated GPUs

Duan Lightfoot

Sr. AI Engineer, Akamai

Duan Lightfoot
Open session

World Congress 2026 North America

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan
Open session

World Congress 2026 North America

Building Stuff with GenAI - The Open Minded Workshop beyond OpenAI

Andreas Erben

CTO for Applied AI and Metaverse at daenet

Andreas Erben
Open session

World Congress 2026 North America

Fast by Design: A Masterclass in High-Performance Web Engineering

Aaron Grogg

Senior web developer, committed to improving the user experience by improving web performance

Aaron Grogg

Related videos

Accelerating Python on GPUs thumbnail
22:18
WWC 2024 β€’ almost 2 years ago

Accelerating Python on GPUs

Paul Graham

Accelerating Python on GPUs thumbnail
24:39
WWC 2025 β€’ 12 months ago

Accelerating Python on GPUs

Paul Graham

CUDA Python: GPU programming for the modern developer thumbnail
29:43
WWC Europe 2026 β€’ about 1 month ago

CUDA Python: GPU programming for the modern developer

Paul Graham

Coffee with Developers - Stephen Jones - NVIDIA thumbnail
37:32
Coffee With Developers β€’ over 1 year ago

Coffee with Developers - Stephen Jones - NVIDIA

Stephen Jones

CUDA in Python thumbnail
21:02
LIVE β€’ over 1 year ago

CUDA in Python

Andy Terrel

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA thumbnail
22:07
WWC 2024 β€’ about 2 years ago

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA

Ankit Patel

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.