WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • May 26, 2021

Serverless deployment of (large) NLP models

Marek Suppa

Can you run massive NLP models within AWS Lambda's strict 250MB limit? Discover how Slido used knowledge distillation and ONNX to achieve sub-100ms serverless inference.

Pause
Mute Enter Fullscreen
#1 about 3 min

Machine learning opportunities within audience interaction platforms

Assessing large text datasets from audience interactions enables automated insights and predictive event intelligence.

#2 about 3 min

Extracting key phrases and analyzing sentiment from audience questions

Applying text preprocessing algorithms reveals event themes and participant emotions without manual oversight.

#3 about 4 min

Detecting similar questions and generating automated quiz responses

Algorithmic grouping of participant questions reduces host redundancy while generative capabilities populate alternative quiz options dynamically.

#4 about 3 min

Moving to serverless deployments for Python machine learning models

Transitioning to serverless architecture bridges integration gaps between a TypeScript platform environment and a Python machine learning stack.

#5 about 3 min

Addressing constraints and limitations of serverless model deployments

Working within serverless functions requires mitigating cold start delays, operating without hardware acceleration, and adapting to fixed file sizes.

#6 about 4 min

Fine-tuning BERT models for audience sentiment analysis classification

Training standard language representations on domain-specific event text improves sentiment scoring accuracy for non-standard sentences.

#7 about 5 min

Reducing model size using knowledge distillation with teacher-student models

Using heavy teacher models to label intermediate datasets trains compact student networks to achieve comparable predictive accuracy strictly inside serverless memory constraints.

#8 about 4 min

Optimizing similar question detection with linear complexity sentence encoders

Replacing standard dual-input classification with isolated pooled vectors drastically decreases computational complexity when comparing live text logs.

#9 about 5 min

Distilling cross-encoder models into smaller efficient sentence embedding models

Leveraging resource-heavy cross-encoder outputs as training subsets for bi-encoders yields micro models capable of continuous semantic matching.

#10 about 5 min

Replacing PyTorch with ONNX runtime for AWS Lambda deployments

Swapping comprehensive deep learning frameworks for a minimalist neural exchange runtime forces overall application payloads below strict Lambda limits.

#11 about 4 min

Evaluating cost effectiveness and summarizing serverless machine learning strategies

Deploying optimized machine learning models via reactive functions proves highly economical against maintaining hosting environments for sporadic traffic.

Matching moments

1:33 min

Scaling model inference leveraging serverless edge functions

Jeremy Murray Jeremy Murray · World Congress 2026 Europe

1:20 min

Reducing infrastructure capacity requirements by deploying small language models

Alexander Wallner Alexander Wallner +3 · World Congress 2024

2:50 min

Executing LoRA fine-tuning using serverless Databricks AI runtimes

Viktoria Semaan Viktoria Semaan · World Congress 2026 Europe

1:45 min

Introduction to serving large language models locally

Patrick Koss Patrick Koss · World Congress 2025

2:44 min

Running lightweight large language models on local hardware

Ekaterina Sirazitdinova · LIVE

5:05 min

Exploring hybrid models, monoliths, and serverless computing

Michael Eisenbart · LIVE

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
14 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
21 days ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
22 days ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
28 days ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude thumbnail
6:16
29 days ago

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude

Chris Heilmann, Daniel Cranney

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas thumbnail
5:46
about 1 month ago

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas

Chris Heilmann, Daniel Cranney, Marc Plogas

Related videos

Unveiling the Magic: Scaling Large Language Models to Serve Millions thumbnail
31:25
World Congress 2025 • about 1 year ago

Unveiling the Magic: Scaling Large Language Models to Serve Millions

Patrick Koss

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML thumbnail
39:58
LIVE • almost 6 years ago

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML

Linda Mohamed

Serverless: Past, Present and Future thumbnail
1:01:02
LIVE • about 6 years ago

Serverless: Past, Present and Future

Oliver Arafat

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40
World Congress 2026 Europe • 3 months ago

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow thumbnail
34:21
LIVE • almost 2 years ago

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01
World Congress 2024 • about 2 years ago

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.