Marek Suppa

Serverless deployment of (large) NLP models

How do you fit a 400MB NLP model into a 250MB serverless function? Learn the model distillation and dependency tricks that make it possible.

Serverless deployment of (large) NLP models
#1about 9 minutes

Exploring practical NLP applications at Slido

Several NLP-powered features are used to enhance user experience, including keyphrase extraction, sentiment analysis, and similar question detection.

#2about 4 minutes

Choosing serverless for ML model deployment

Serverless was chosen for its ease of deployment and minimal maintenance, but it introduces challenges like cold starts and strict package size limits.

#3about 8 minutes

Shrinking large BERT models for sentiment analysis

Knowledge distillation is used to train smaller, faster models like TinyBERT from a large, fine-tuned BERT base model without significant performance loss.

#4about 8 minutes

Building an efficient similar question detection model

Sentence-BERT (SBERT) provides an efficient alternative to standard BERT for semantic similarity, and knowledge distillation helps create smaller, deployable versions.

#5about 3 minutes

Using ONNX Runtime for lightweight model inference

The large PyTorch library is replaced with the much smaller ONNX Runtime to fit the model and its dependencies within AWS Lambda's package size limits.

#6about 3 minutes

Analyzing serverless ML performance and cost-effectiveness

Increasing allocated RAM for a Lambda function improves inference speed, potentially making serverless more cost-effective than a dedicated server for uneven workloads.

#7about 3 minutes

Key takeaways for deploying NLP models serverlessly

Successful serverless deployment of large NLP models requires aggressive model size reduction, lightweight inference libraries, and an understanding of the platform's limitations.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

Organizing a developer conference for 15,000 attendees
01:32 MIN

Organizing a developer conference for 15,000 attendees

Cat Herding with Lions and Tigers - Christian Heilmann

Automating video post-production with local scripts
02:54 MIN

Automating video post-production with local scripts

Cat Herding with Lions and Tigers - Christian Heilmann

Rapid-fire thoughts on the future of work
02:44 MIN

Rapid-fire thoughts on the future of work

What 2025 Taught Us: A Year-End Special with Hung Lee

Moving beyond headcount to solve business problems
04:27 MIN

Moving beyond headcount to solve business problems

What 2025 Taught Us: A Year-End Special with Hung Lee

Why corporate AI adoption lags behind the hype
03:28 MIN

Why corporate AI adoption lags behind the hype

What 2025 Taught Us: A Year-End Special with Hung Lee

Breaking down silos between HR, tech, and business
03:39 MIN

Breaking down silos between HR, tech, and business

What 2025 Taught Us: A Year-End Special with Hung Lee

Why HR struggles with technology implementation and adoption
04:22 MIN

Why HR struggles with technology implementation and adoption

What 2025 Taught Us: A Year-End Special with Hung Lee

Featured Partners

Related Videos

See all videos
Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML 39:58

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML

Linda Mohamed

Unveiling the Magic: Scaling Large Language Models to Serve Millions31:25

Unveiling the Magic: Scaling Large Language Models to Serve Millions

Patrick Koss

Multilingual NLP pipeline up and running from scratch52:37

Multilingual NLP pipeline up and running from scratch

Kateryna Hrytsaienko

End the Monolith! Lessons learned adopting Serverless23:19

End the Monolith! Lessons learned adopting Serverless

Nočnica Fee

From ML to LLM: On-device AI in the Browser27:23

From ML to LLM: On-device AI in the Browser

Nico Martin

Self-Hosted LLMs: From Zero to Inference30:04

Self-Hosted LLMs: From Zero to Inference

Roberto Carratalá & Cedric Clyburn

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

Server Side Serverless in Swift48:07

Server Side Serverless in Swift

Sebastien Stormacq

Related Articles

View all articles
BB
Benedikt Bischof
MLOps – What’s the deal behind it?
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine learning veteran. Starting back in 2014 he observed ...
MLOps – What’s the deal behind it?

From learning to earning

Jobs that call for the skills explored in this talk.