Ron Dagdag

Making neural networks portable with ONNX

What if your ML models were as portable as a PDF? Learn how ONNX creates a universal standard to run any model on any hardware, even in the browser.

Making neural networks portable with ONNX
#1about 6 minutes

Understanding ONNX as a portable format for ML models

Machine learning models are made portable across different frameworks and hardware using the ONNX open standard, similar to how PDF works for documents.

#2about 2 minutes

When to use ONNX for your machine learning projects

ONNX is ideal for deploying models across different programming languages, achieving low-latency inferencing, and running on resource-constrained edge or IoT devices.

#3about 12 minutes

Four methods for creating or acquiring ONNX models

Models can be obtained from the ONNX Model Zoo, built with tools like Azure Custom Vision, converted from existing frameworks like PyTorch, or used as an intermediary format.

#4about 7 minutes

Deploying models with the high-performance ONNX Runtime

The ONNX Runtime is a high-performance inference engine for deploying models to the cloud or edge devices, bridging the gap between data science and production software engineering.

#5about 4 minutes

Running an ONNX model in a Node.js application

A practical demonstration shows how to load an ONNX model and perform inference within a server-side Node.js application using the `onnxruntime-node` package.

#6about 9 minutes

Performing inference in the browser with ONNX Runtime Web

An emotion detection model is run directly in the browser using ONNX Runtime Web, showcasing client-side inference with JavaScript for privacy and offline capability.

#7about 3 minutes

Optimizing ONNX models for mobile and React Native

ONNX Runtime Mobile provides a lightweight solution for iOS and Android by converting models to a pre-optimized `.ort` format for smaller binary sizes.

#8about 8 minutes

Q&A on starting a career in machine learning

Advice is given on how software developers can enter the machine learning field by starting with model integration and deployment before diving deep into model creation.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Deploying machine learning models in a JavaScript environment
23:20 MIN

Deploying machine learning models in a JavaScript environment

Shoot for the moon - machine learning for automated online ad detection

Understanding the ONNX format for model interoperability
44:19 MIN

Understanding the ONNX format for model interoperability

Introduction to Azure Machine Learning

Consuming an ONNX model in a .NET console application
38:21 MIN

Consuming an ONNX model in a .NET console application

Machine Learning in ML.NET

The technical challenges of running LLMs in browsers
09:43 MIN

The technical challenges of running LLMs in browsers

From ML to LLM: On-device AI in the Browser

Using ONNX Runtime for lightweight model inference
30:21 MIN

Using ONNX Runtime for lightweight model inference

Serverless deployment of (large) NLP models

The future of on-device AI hardware and APIs
23:20 MIN

The future of on-device AI hardware and APIs

From ML to LLM: On-device AI in the Browser

Matching edge AI challenges with NVIDIA's solutions
27:27 MIN

Matching edge AI challenges with NVIDIA's solutions

Trends, Challenges and Best Practices for AI at the Edge

Simplifying development with high-level AI frameworks
18:23 MIN

Simplifying development with high-level AI frameworks

Privacy-first in-browser Generative AI web apps: offline-ready, future-proof, standards-based

Featured Partners

Related Videos

See all videos
Privacy-first in-browser Generative AI web apps: offline-ready, future-proof, standards-based30:13

Privacy-first in-browser Generative AI web apps: offline-ready, future-proof, standards-based

Maxim Salnikov

Generative AI power on the web: making web apps smarter with WebGPU and WebNN32:27

Generative AI power on the web: making web apps smarter with WebGPU and WebNN

Christian Liebel

Prompt API & WebNN: The AI Revolution Right in Your Browser30:36

Prompt API & WebNN: The AI Revolution Right in Your Browser

Christian Liebel

From ML to LLM: On-device AI in the Browser27:23

From ML to LLM: On-device AI in the Browser

Nico Martin

Trends, Challenges and Best Practices for AI at the Edge29:50

Trends, Challenges and Best Practices for AI at the Edge

Ekaterina Sirazitdinova

How AI Models Get Smarter35:16

How AI Models Get Smarter

Ankit Patel

Your Next AI Needs 10,000 GPUs. Now What?29:52

Your Next AI Needs 10,000 GPUs. Now What?

Anshul Jindal & Martin Piercy

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA22:07

WWC24 - Ankit Patel - Unlocking the Future Breakthrough Application Performance and Capabilities with NVIDIA

Ankit Patel

From learning to earning

Jobs that call for the skills explored in this talk.