> Markdown version of [/videos/588-using-webassembly-for-in-database-machine-learning?t=259](https://www.wearedevelopers.com/videos/588-using-webassembly-for-in-database-machine-learning?t=259). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Using WebAssembly for in-database Machine Learning Tired of building expensive ETL pipelines just to run machine learning models? Learn how WebAssembly lets you execute secure, near-native ML workloads directly inside your database. - **Speakers:** [Akmal Chaudhri](https://www.wearedevelopers.com/@akmal-chaudhri) - **Event:** WeAreDevelopers LIVE - **Published:** June 1, 2023 - **Duration:** 58:53 - **URL:** https://www.wearedevelopers.com/videos/588-using-webassembly-for-in-database-machine-learning ## Summary Executing machine learning workloads directly within a database traditionally poses performance and security challenges. WebAssembly introduces a transformative approach by allowing developers to write code in languages like Rust or C++ and execute it securely as a user-defined function inside a database management system. This architectural shift enables data and code co-location, eliminating the need for expensive ETL pipelines to external machine learning frameworks while running at near-native speeds in an isolated sandbox. Using SingleStore as an example, developers can leverage the WebAssembly SDK, Rust toolchain, and tools like PushWasm to compile and deploy custom logic directly into the database. By defining inputs and outputs with an interface definition file, developers can wrap existing open-source libraries—such as the VADER sentiment analysis tool—into a modular unit. Once loaded, the database treats this module as a standard procedural function, allowing users to run complex inferences, like analyzing the sentiment of a large movie review dataset, using standard queries. Bringing computations to the data maximizes efficiency, especially within distributed environments where data movement creates massive overhead. While the ecosystem is still maturing—currently characterized by manual setup processes, limited IDE integration, and a focus on compiled languages over dynamic ones like Python—initiatives from the Bytecode Alliance are rapidly standardizing these workflows. Ultimately, utilizing WebAssembly for in-database machine learning empowers organizations to securely extend native capabilities, unlocking high-performance analytics without compromising system stability. **Keywords:** webassembly, in-database machine learning, singlestore, user-defined functions, rust programming, data co-location, sentiment analysis, bytecode alliance, sandbox execution, etl elimination, interface definition files, pushwasm tool, vader algorithm, distributed databases, predictive scoring models ## Chapters 1. **Introduction to WebAssembly for in-database machine learning** (00:05) — An overview of utilizing WebAssembly to execute machine learning modules directly within a database management system. 1. **Evolution of distributed SQL database management systems** (02:26) — How modern distributed relational databases enable horizontal scaling for analytical and operational workloads. 1. **Options for database machine learning integration architectures** (04:19) — Utilizing tools like Apache Spark, Python libraries, and vector embeddings to perform machine learning near or within databases. 1. **Benefits of executing WebAssembly natively in databases** (07:15) — How WebAssembly addresses complex procedural SQL, provides data co-location, and reduces developer effort for data science tasks. 1. **Architectural flow of WebAssembly inside a DBMS** (10:30) — The process of compiling business logic in languages like Rust or C++ and executing it within a secure database sandbox. 1. **Setting up the Wasm SDK and Rust toolchain** (13:55) — Instructions for downloading and configuring the WebAssembly SDK and Bytecode Alliance tools for local development. 1. **Preparing local tools to push Wasm executable modules** (16:38) — Cloning and building the push-wasm GitHub dependency tool to bridge compiled modules into SingleStore. 1. **Defining Wit interfaces and declaring Rust backend dependencies** (18:39) — Configuring an interface definition file to specify input and output types for a VADER sentiment algorithm. 1. **Compiling and uploading Rust logic into the DBMS** (21:29) — Creating the actual sentiment scoring function in Rust and pushing it to the database for use as a user-defined function. 1. **Testing sentiment inference behavior via SQL select statements** (24:37) — Verifying the WebAssembly function behavior on text patterns like capitalization and exclamation points directly inside standard SQL queries. 1. **Live demo applying sentiment analysis on an IMDB dataset** (27:00) — Applying the deployed Rust WebAssembly module across an IMDB review table to batch-compute machine learning scores without ETL. 1. **Key takeaways and developer resources for WebAssembly integration** (32:39) — A summary of WebAssembly extensibility capabilities along with resources from the Bytecode Alliance. 1. **Audience Q&A on language support, performance, and tradeoffs** (36:20) — A community discussion focusing on dynamic language support, native performance comparisons, testing workflows, and potential cloud provider implementations. ## Related Moments - [Introduction to WebAssembly in a cloud computing context](https://www.wearedevelopers.com/videos/972-webassembly-the-next-frontier-of-cloud-computing) (from "WebAssembly: The Next Frontier of Cloud Computing") - [Introduction to WebAssembly as a compilation target](https://www.wearedevelopers.com/videos/652-using-webassembly-to-run-extend-and-secure-your-application) (from "Using WebAssembly to run, extend, and secure your application") - [Applying WebAssembly to serverless cloud environments](https://www.wearedevelopers.com/videos/687-the-future-of-cloud-is-webassembly) (from "The Future of Cloud is WebAssembly") - [Redefining WebAssembly beyond browser video games](https://www.wearedevelopers.com/videos/886-webassembly-the-next-frontier-of-cloud-computing) (from "WebAssembly: The Next Frontier of Cloud Computing") - [Early implementations of WebAssembly across major industries](https://www.wearedevelopers.com/videos/652-using-webassembly-to-run-extend-and-secure-your-application) (from "Using WebAssembly to run, extend, and secure your application") - [Using WebAssembly for fast and secure serverless computing](https://www.wearedevelopers.com/videos/673-fun-with-paas-how-to-use-cloud-foundry-and-its-uniqueness-in-creative-ways) (from "Fun with PaaS – How to use Cloud Foundry and its uniqueness in creative ways") ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Building AI Solutions with Rust and Docker](https://www.wearedevelopers.com/magazine/494-building-ai-solutions-with-rust-and-docker) - [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) ## Related Jobs - [Data Scientist](https://www.wearedevelopers.com/jobs/ext/1351648-data-scientist) at **Almedia** - [Principal Software Engineer, Enterprise AI Platform](https://www.wearedevelopers.com/jobs/ext/1467292-principal-software-engineer-enterprise-ai-platform) at **GitHub** - [Principal Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1706410-principal-machine-learning-engineer) at **Almedia** - [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat** - [Principal Engineer - AI Search & Vector Infrastructure](https://www.wearedevelopers.com/jobs/ext/381484-principal-engineer-ai-search-vector-infrastructure) at **Redis** - [Data & Machine Learning Engineer | Hybrid work](https://www.wearedevelopers.com/jobs/ext/431779-data-machine-learning-engineer-hybrid-work) at **SMG Swiss Marketplace Group**