Kateryna Hrytsaienko

Multilingual NLP pipeline up and running from scratch

Translating text can reduce your NLP model's accuracy by 20%. Learn how to build a single, unified pipeline that handles multiple languages without the loss.

Multilingual NLP pipeline up and running from scratch
#1about 3 minutes

The challenge of building end-to-end NLP pipelines

There is a lack of comprehensive guides for integrating multilingual NLP models into applications with proper CI/CD practices, especially for non-English languages.

#2about 5 minutes

Understanding the core components of an NLP pipeline

A typical NLP pipeline consists of three key stages: pre-processing, feature extraction, and modeling, with pre-processing being critical for handling unstructured data.

#3about 8 minutes

Why simply translating everything to English is not enough

Translating all text to English for NLP analysis can decrease accuracy by up to 20% due to lost semantic nuance and dialectical differences.

#4about 10 minutes

Generalizing languages with stemming and bag-of-words

Handle similar languages by using stemming to find common root words and a bag-of-words model with a similarity index to treat them as a single language.

#5about 5 minutes

Achieving high accuracy with a unified language model

By training classifiers on stemmed and normalized vectors from multiple similar languages, it's possible to achieve high accuracy of around 90% in tasks like topic classification.

#6about 8 minutes

Choosing the right deployment strategy for your model

Decide between embedding your model or exposing it as an API, considering options like serverless for simple cases or Kubernetes for scalable, cloud-agnostic deployments.

#7about 7 minutes

Implementing a CI/CD pipeline for your NLP model

Establish an MLOps workflow with continuous training, integration, and delivery by containerizing your model with Docker and automating builds with tools like GitHub Actions.

#8about 6 minutes

Q&A on slang processing, debugging, and transformers

The Q&A covers practical advice on handling slang with dictionaries, debugging with robust logging, and understanding the complexity gap between traditional methods and transformers like BERT.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

Organizing a developer conference for 15,000 attendees
01:32 MIN

Organizing a developer conference for 15,000 attendees

Cat Herding with Lions and Tigers - Christian Heilmann

Using content channels to build an event community
04:49 MIN

Using content channels to build an event community

Cat Herding with Lions and Tigers - Christian Heilmann

Selecting strategic partners and essential event tools
03:17 MIN

Selecting strategic partners and essential event tools

Cat Herding with Lions and Tigers - Christian Heilmann

Why corporate AI adoption lags behind the hype
03:28 MIN

Why corporate AI adoption lags behind the hype

What 2025 Taught Us: A Year-End Special with Hung Lee

Rapid-fire thoughts on the future of work
02:44 MIN

Rapid-fire thoughts on the future of work

What 2025 Taught Us: A Year-End Special with Hung Lee

Moving beyond headcount to solve business problems
04:27 MIN

Moving beyond headcount to solve business problems

What 2025 Taught Us: A Year-End Special with Hung Lee

The future of recruiting beyond talent acquisition
03:15 MIN

The future of recruiting beyond talent acquisition

What 2025 Taught Us: A Year-End Special with Hung Lee

Featured Partners

Related Videos

See all videos
A beginner’s guide to modern natural language processing56:46

A beginner’s guide to modern natural language processing

Jodie Burchell

Python-Based Data Streaming Pipelines Within Minutes39:04

Python-Based Data Streaming Pipelines Within Minutes

Bobur Umurzokov

Creating Industry ready solutions with LLM Models58:00

Creating Industry ready solutions with LLM Models

Vijay Krishan Gupta & Gauravdeep Singh Lotey

Multimodal Generative AI Demystified56:55

Multimodal Generative AI Demystified

Ekaterina Sirazitdinova

Overview of Machine Learning in Python57:46

Overview of Machine Learning in Python

Adrian Schmitt

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow34:21

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

From Traction to Production: Maturing your LLMOps step by step41:45

From Traction to Production: Maturing your LLMOps step by step

Maxim Salnikov

The state of MLOps - machine learning in production at enterprise scale47:19

The state of MLOps - machine learning in production at enterprise scale

Bas Geerdink 

Related Articles

View all articles
CH
Chris Heilmann
With AIs wide open - WeAreDevelopers at All Things Open 2025
Last week our VP of Developer Relations, Chris Heilmann, flew to Raleigh, North Carolina to present at All Things Open . An excellent event he had spoken at a few times in the past and this being the “Lucky 13” edition, he didn’t hesitate to come and...
With AIs wide open - WeAreDevelopers at All Things Open 2025

From learning to earning

Jobs that call for the skills explored in this talk.