WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Dec 16, 2024

From ML to LLM: On-device AI in the Browser

Nico Martin

Run large language models entirely in the browser without making a single cloud request. Discover how WebGPU, 4-bit quantization, and local RAG unlock fast, privacy-preserving AI applications.

Pause
Mute Enter Fullscreen
#1 about 3 min

Overcoming limitations of browser speech detection for local dialects

Attempting to transcribe unique regional languages highlights the limitations of standard built-in browser speech APIs.

#2 about 2 min

Accelerating machine learning in the browser using WebGPU

Processing heavy mathematical operations requires efficient backends like WebGPU over standard CPU or WebAssembly.

#3 about 2 min

Comparing hardware acceleration for face landmark detection

Drawing face meshes in real-time demonstrates the framerate differences between CPU, WebGL, and WebGPU execution.

#4 about 2 min

Building interactive browser extensions using hand gesture recognition

Tracking index and thumb finger positions allows for pinch gestures that map to scrolling and clicking on websites.

#5 about 3 min

Detecting custom speech commands locally with trained models

Recording and training audio samples entirely on the client side enables the detection of unsupported regional dialects.

#6 about 3 min

Mechanics of running large language models in browsers

Converting text input to numerical tokens and managing billions of model weights is necessary for local language model execution.

#7 about 1 min

Shrinking language models for browsers using four-bit quantization

Storing weights as four-bit floats instead of higher precision formats reduces model payload sizes to acceptable browser limits.

#8 about 2 min

Compiling language models to WebAssembly using web libraries

Utilizing open-source tools like WebLLM and Apache TVM converts standard architectures into browser-compatible WebAssembly engines.

#9 about 3 min

Integrating local language models into progressive web apps

Loading smaller parameter models in web applications processes input tokens without relying on external cloud providers.

#10 about 6 min

Fact-checking language models using retrieval-augmented generation

Parsing documents and creating vector embeddings directly in the browser ensures answers are based on verifiable local contexts.

#11 about 4 min

Adopting local AI as a privacy-preserving progressive enhancement

Optimizing for diverse user hardware requires leveraging upcoming web neural network APIs while keeping AI features optional.

Matching moments

7:17 min

Accelerating in-browser AI with WebGPU and WebNN

Christian Liebel Christian Liebel · World Congress 2024

2:46 min

The case for native AI in web browsers

Maxim Salnikov Maxim Salnikov · World Congress 2025

1:04 min

Running local large language models securely using paired web GPUs

Önder Ceylan Önder Ceylan · World Congress 2024

2:17 min

Accelerating local machine learning models via WebNN

Christian Liebel Christian Liebel · World Congress 2026 Europe

2:54 min

Leveraging Chrome AI and nano models for web applications

Raymond Camden · Perf + AI

1:33 min

Real-world implementations of browser-based machine learning

Håkan Silfvernagel · LIVE

More from WeAreDevelopers LIVE

Fake or News - Claude's DNA Discovery, Quick Cloning & Crime on Insta thumbnail
6:07
1 day ago

Fake or News - Claude's DNA Discovery, Quick Cloning & Crime on Insta

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40
1 day ago

WeAreDevelopers LIVE - Playing in Production

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
15 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
22 days ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
24 days ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
29 days ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Related videos

Generative AI power on the web: making web apps smarter with WebGPU and WebNN thumbnail
32:27
World Congress 2024 • about 2 years ago

Generative AI power on the web: making web apps smarter with WebGPU and WebNN

Christian Liebel

Exploring the Future of Web AI with Google thumbnail
33:51
Coffee With Developers • over 1 year ago

Exploring the Future of Web AI with Google

Thomas Steiner

Prompt API & WebNN: The AI Revolution Right in Your Browser thumbnail
30:36
World Congress 2025 • about 1 year ago

Prompt API & WebNN: The AI Revolution Right in Your Browser

Christian Liebel

AI in Your Browser: Exploring Chrome's Built-In LLM thumbnail
29:59
World Congress 2026 Europe • 3 months ago

AI in Your Browser: Exploring Chrome's Built-In LLM

Daniel Ostrovsky

How Web AI Can Power the Agentic Web - Jason Mayes (Google) thumbnail
20:57
Coffee With Developers • 4 months ago

How Web AI Can Power the Agentic Web - Jason Mayes (Google)

Jason Mayes

Privacy-first in-browser Generative AI web apps: offline-ready, future-proof, standards-based thumbnail
30:13
World Congress 2025 • about 1 year ago

Privacy-first in-browser Generative AI web apps: offline-ready, future-proof, standards-based

Maxim Salnikov

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.