WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Jun 17, 2020

30 Golden Rules of Deep Learning Performance

Anirudh Koul

Stop buying more compute power to fix GPU starvation. Discover how serializing datasets, shifting augmentation, and using mixed precision can exponentially accelerate your deep learning pipelines.

Pause
Mute Enter Fullscreen
#1 about 7 min

Identifying major training inefficiencies in machine learning processes

Waiting for large neural networks to finish training results in wasted engineering cycles and excessive infrastructure costs.

#2 about 3 min

Speeding up model training cycles with transfer learning techniques

Reusing pre-trained convolutional neural weights directly avoids initializing large multi-layer tasks from computational scratch.

#3 about 5 min

Diagnosing training performance bottlenecks with visual profiling tools

Identifying CPU-starved graphic processors is possible using native tensorboard interfaces and terminal device queries.

#4 about 4 min

Optimizing custom dataset preparation to avoid file reading latency

Serializing custom input datasets into chunked file structures naturally prevents I/O choking while reading network files.

#5 about 5 min

Building asynchronous data pipelines with TensorFlow data APIs

Interleaving batch fetching inside background threads ensures the active GPU is constantly being fed data arrays.

#6 about 3 min

Offloading complex image pre-processing and augmentations to GPUs

Porting image manipulation procedures away from main application threads explicitly removes primary CPU processing overheads.

#7 about 5 min

Maximizing GPU utilization via ideal batch sizes and mixed precision

Adopting minimum size variables alongside properly aligned layer multiples enables aggressive tensor core processing yields.

#8 about 2 min

Finding the optimal learning rate automatically to simplify tuning

Running cyclic learning algorithms across isolated experimental trials algorithmically narrows the range of potential validation metrics.

#9 about 2 min

Boosting small loop logic performance by compiling execution graphs

Decorating standard Python callbacks internally compiles backend graphs instead of explicitly interpreting local environment sequences.

#10 about 3 min

Accelerating slow convergence loops using progressive image resizing techniques

Scaling up training complexities gradually through varying image resolutions reduces early step training spans massively.

#11 about 2 min

Leveraging specialized software environments optimized for local hardware specs

Routing default open source dependencies through optimized vendor streams yields instant processor vectorized performance benefits.

#12 about 6 min

Distributing local models across cloud environments and multi-GPU frameworks

Pushing batch workloads to cloud virtual machines drastically offsets expensive single node upfront hardware architectures.

#13 about 3 min

Choosing compact predefined neural network architectures for production environments

Committing to benchmarked open source models prevents memory overflow crashes on severely limited end user devices.

#14 about 1 min

Shrinking compiled models using integer quantization for inference edges

Downgrading compiled float networks to minimal integer values significantly shrinks deployment sizes without major capability drops.

#15 about 2 min

Consolidating software strategies and reviewing deep learning performance resources

Reviewing optimization codes provides developers a comprehensive single sheet integration pipeline for immediate model scale factors.

Matching moments

4:58 min

Training neural networks and switching hardware inside notebooks

Qiyang Duan · LIVE

2:00 min

Cost and power economics of hardware acceleration

Ankit Patel Ankit Patel · WWC 2024

1:49 min

Transformers and highly scalable parallel processing

Kamen Petroff Kamen Petroff · WWC Europe 2026

2:47 min

Performance benchmarks and Android developer adoption

Gian Marco Iodice Gian Marco Iodice · WWC 2025

2:52 min

Scaling model training for large datasets

Adrian Schmitt · LIVE

6:41 min

Optimizing edge model throughput to ensure real-time performance

Ekaterina Sirazitdinova · WWC 2023

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
5 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
5 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
12 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
19 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
26 days ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail
5:55
about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Agents That Own Their Inference: Building Production AI Agents on Dedicated GPUs

Duan Lightfoot

Sr. AI Engineer, Akamai

Duan Lightfoot
Open session

World Congress 2026 North America

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan
Open session

World Congress 2026 North America

Compute for your AI model: GPUs, LPUs, TPUs and beyond..

Kushaagra Goyal

Tech Lead at Rubrik, ex-CTO at Gan.AI, ex-Databricks

Kushaagra Goyal
Open session

World Congress 2026 North America

Trust, But Verify: Continuous GPU Validation at Scale

Kyle Bell

VP of AI @ TensorWave

Kyle Bell
Open session

World Congress 2026 North America

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization

Cedric Clyburn, Legare Kerrison

Cedric Clyburn
Legare Kerrison
Open session

World Congress 2026 North America

Understanding LLM Architectures: Inside the Design of Modern Models

Jofia Jose Prakash

Enterprise AI Architect at American Chemical Society

Jofia Jose Prakash

Related videos

Accelerating Python on GPUs thumbnail
59:43
LIVE • over 2 years ago

Accelerating Python on GPUs

Paul Graham

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40
WWC Europe 2026 • about 1 month ago

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl

Trends, Challenges and Best Practices for AI at the Edge thumbnail
29:50
WWC 2023 • almost 3 years ago

Trends, Challenges and Best Practices for AI at the Edge

Ekaterina Sirazitdinova

Accelerating Python on GPUs thumbnail
22:18
WWC 2024 • almost 2 years ago

Accelerating Python on GPUs

Paul Graham

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01
WWC 2024 • almost 2 years ago

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA thumbnail
22:07
WWC 2024 • about 2 years ago

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA

Ankit Patel

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.