WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • May 26, 2021

Serverless deployment of (large) NLP models

Marek Suppa

Can you run massive NLP models within AWS Lambda's strict 250MB limit? Discover how Slido used knowledge distillation and ONNX to achieve sub-100ms serverless inference.

Pause
Mute Enter Fullscreen
#1 about 3 min

Machine learning opportunities within audience interaction platforms

Assessing large text datasets from audience interactions enables automated insights and predictive event intelligence.

#2 about 3 min

Extracting key phrases and analyzing sentiment from audience questions

Applying text preprocessing algorithms reveals event themes and participant emotions without manual oversight.

#3 about 4 min

Detecting similar questions and generating automated quiz responses

Algorithmic grouping of participant questions reduces host redundancy while generative capabilities populate alternative quiz options dynamically.

#4 about 3 min

Moving to serverless deployments for Python machine learning models

Transitioning to serverless architecture bridges integration gaps between a TypeScript platform environment and a Python machine learning stack.

#5 about 3 min

Addressing constraints and limitations of serverless model deployments

Working within serverless functions requires mitigating cold start delays, operating without hardware acceleration, and adapting to fixed file sizes.

#6 about 4 min

Fine-tuning BERT models for audience sentiment analysis classification

Training standard language representations on domain-specific event text improves sentiment scoring accuracy for non-standard sentences.

#7 about 5 min

Reducing model size using knowledge distillation with teacher-student models

Using heavy teacher models to label intermediate datasets trains compact student networks to achieve comparable predictive accuracy strictly inside serverless memory constraints.

#8 about 4 min

Optimizing similar question detection with linear complexity sentence encoders

Replacing standard dual-input classification with isolated pooled vectors drastically decreases computational complexity when comparing live text logs.

#9 about 5 min

Distilling cross-encoder models into smaller efficient sentence embedding models

Leveraging resource-heavy cross-encoder outputs as training subsets for bi-encoders yields micro models capable of continuous semantic matching.

#10 about 5 min

Replacing PyTorch with ONNX runtime for AWS Lambda deployments

Swapping comprehensive deep learning frameworks for a minimalist neural exchange runtime forces overall application payloads below strict Lambda limits.

#11 about 4 min

Evaluating cost effectiveness and summarizing serverless machine learning strategies

Deploying optimized machine learning models via reactive functions proves highly economical against maintaining hosting environments for sporadic traffic.

Matching moments

1:33 min

Scaling model inference leveraging serverless edge functions

Jeremy Murray Jeremy Murray · WWC Europe 2026

1:20 min

Reducing infrastructure capacity requirements by deploying small language models

Alexander Wallner Alexander Wallner +3 · WWC 2024

2:50 min

Executing LoRA fine-tuning using serverless Databricks AI runtimes

Viktoria Semaan Viktoria Semaan · WWC Europe 2026

1:45 min

Introduction to serving large language models locally

Patrick Koss Patrick Koss · WWC 2025

2:44 min

Running lightweight large language models on local hardware

Ekaterina Sirazitdinova · LIVE

5:05 min

Exploring hybrid models, monoliths, and serverless computing

Michael Eisenbart · LIVE

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
5 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
5 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
12 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
19 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
26 days ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail
5:55
about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization

Legare Kerrison, Cedric Clyburn

Legare Kerrison
Cedric Clyburn
Open session

World Congress 2026 North America

No Single Model to Rule Them All: Building Resilient AI Agents Across Open & Closed LLMs

Emmanuel Acheampong

Senior Manager Developer Relations at Crusoe AI

Emmanuel Acheampong
Open session

World Congress 2026 North America

Run your agents in Kubernetes: Build once, deploy anywhere. But really?

Michal Salanci

Senior Systems Engineer at ESET Cybersecurity

Michal Salanci
Open session

World Congress 2026 North America

The $0.15/month Algo-Trader: Architecting High-Performance Serverless Trading Engines

Vishwak Thatikonda

Lead Full Stack Engineer at SDH Systems

Vishwak Thatikonda
Open session

World Congress 2026 North America

Headroom: A Context Optimization Layer for LLM Applications

Tejas Chopra

Senior Software Engineer at Netflix

Tejas Chopra
Open session

World Congress 2026 North America

Agents That Own Their Inference: Building Production AI Agents on Dedicated GPUs

Duan Lightfoot

Sr. AI Engineer, Akamai

Duan Lightfoot

Related videos

Unveiling the Magic: Scaling Large Language Models to Serve Millions thumbnail
31:25
WWC 2025 • 12 months ago

Unveiling the Magic: Scaling Large Language Models to Serve Millions

Patrick Koss

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML thumbnail
39:58
LIVE • over 5 years ago

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML

Linda Mohamed

Serverless: Past, Present and Future thumbnail
1:01:02
LIVE • almost 6 years ago

Serverless: Past, Present and Future

Oliver Arafat

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40
WWC Europe 2026 • about 1 month ago

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow thumbnail
34:21
LIVE • almost 2 years ago

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01
WWC 2024 • almost 2 years ago

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.