WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Dec 16, 2024

From ML to LLM: On-device AI in the Browser

Nico Martin

Run large language models entirely in the browser without making a single cloud request. Discover how WebGPU, 4-bit quantization, and local RAG unlock fast, privacy-preserving AI applications.

Pause
Mute Enter Fullscreen
#1 about 3 min

Overcoming limitations of browser speech detection for local dialects

Attempting to transcribe unique regional languages highlights the limitations of standard built-in browser speech APIs.

#2 about 2 min

Accelerating machine learning in the browser using WebGPU

Processing heavy mathematical operations requires efficient backends like WebGPU over standard CPU or WebAssembly.

#3 about 2 min

Comparing hardware acceleration for face landmark detection

Drawing face meshes in real-time demonstrates the framerate differences between CPU, WebGL, and WebGPU execution.

#4 about 2 min

Building interactive browser extensions using hand gesture recognition

Tracking index and thumb finger positions allows for pinch gestures that map to scrolling and clicking on websites.

#5 about 3 min

Detecting custom speech commands locally with trained models

Recording and training audio samples entirely on the client side enables the detection of unsupported regional dialects.

#6 about 3 min

Mechanics of running large language models in browsers

Converting text input to numerical tokens and managing billions of model weights is necessary for local language model execution.

#7 about 1 min

Shrinking language models for browsers using four-bit quantization

Storing weights as four-bit floats instead of higher precision formats reduces model payload sizes to acceptable browser limits.

#8 about 2 min

Compiling language models to WebAssembly using web libraries

Utilizing open-source tools like WebLLM and Apache TVM converts standard architectures into browser-compatible WebAssembly engines.

#9 about 3 min

Integrating local language models into progressive web apps

Loading smaller parameter models in web applications processes input tokens without relying on external cloud providers.

#10 about 6 min

Fact-checking language models using retrieval-augmented generation

Parsing documents and creating vector embeddings directly in the browser ensures answers are based on verifiable local contexts.

#11 about 4 min

Adopting local AI as a privacy-preserving progressive enhancement

Optimizing for diverse user hardware requires leveraging upcoming web neural network APIs while keeping AI features optional.

Matching moments

7:17 min

Accelerating in-browser AI with WebGPU and WebNN

Christian Liebel Christian Liebel · World Congress 2024

2:46 min

The case for native AI in web browsers

Maxim Salnikov Maxim Salnikov · World Congress 2025

1:04 min

Running local large language models securely using paired web GPUs

Önder Ceylan Önder Ceylan · World Congress 2024

1:52 min

Industry alignment on running language models in browsers

Daniel Ostrovsky Daniel Ostrovsky · World Congress 2026 North America

2:17 min

Accelerating local machine learning models via WebNN

Christian Liebel Christian Liebel · World Congress 2026 Europe

2:54 min

Leveraging Chrome AI and nano models for web applications

Raymond Camden · Perf + AI

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Tech Education in Two Languages thumbnail
1:00:18
4 days ago

WeAreDevelopers LIVE - Tech Education in Two Languages

Daniel Cranney, Roxy Rodriguez-Becker

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40
11 days ago

WeAreDevelopers LIVE - Playing in Production

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
25 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
about 1 month ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
about 1 month ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
about 1 month ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Upcoming sessions on this topic

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Open session

React Under Fire: Rendering Performance for Streaming Agent UIs

  • Abhirupa Mitra

    Slack

    Senior Member of Technical Staff

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Open session

ULTRON: Your Agent Has No Hands - Real-Time Voice, MCP, and Ten Phones on Stage

  • Sagar Tamang

    Two Spoon Technologies

    AI Engineer

Open session

Thinking in Graphs, Reasoning with Gemini!

  • Bhavesh Bhatt

    N/A

    Lead AI Engineer

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Related videos

Generative AI power on the web: making web apps smarter with WebGPU and WebNN thumbnail
32:27
World Congress 2024 • about 2 years ago

Generative AI power on the web: making web apps smarter with WebGPU and WebNN

Christian Liebel

Exploring the Future of Web AI with Google thumbnail
33:51
Coffee With Developers • over 1 year ago

Exploring the Future of Web AI with Google

Thomas Steiner

Prompt API & WebNN: The AI Revolution Right in Your Browser thumbnail
30:36
World Congress 2025 • about 1 year ago

Prompt API & WebNN: The AI Revolution Right in Your Browser

Christian Liebel

AI in Your Browser: Exploring Chrome's Built-In LLM thumbnail
29:59
World Congress 2026 Europe • 3 months ago

AI in Your Browser: Exploring Chrome's Built-In LLM

Daniel Ostrovsky

Small LLM in your Browser: Huge Opportunities for Web Applications thumbnail
31:28
World Congress 2026 North America • 15 days ago

Small LLM in your Browser: Huge Opportunities for Web Applications

Daniel Ostrovsky

How Web AI Can Power the Agentic Web - Jason Mayes (Google) thumbnail
20:57
Coffee With Developers • 5 months ago

How Web AI Can Power the Agentic Web - Jason Mayes (Google)

Jason Mayes

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.