WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Feb 23, 2024

A beginner’s guide to modern natural language processing

Jodie Burchell

Stop wasting compute on expensive transformers. Basic count vectorization often rivals BERT for simple classification tasks. Discover how to match model complexity to your dataset realities.

Pause
Mute Enter Fullscreen
#1 about 3 min

Introduction to natural language processing challenges

Processing unstructured text requires robust strategies to successfully handle grammatical relationships and contextual ambiguities.

#2 about 3 min

Comparing tabular data and unstructured text inputs

Machine learning algorithms require structured numerical features instead of variable-length raw text strings.

#3 about 3 min

Introduction to fundamental text vectorization concepts

Transforming raw phrases into numeric feature columns allows machine learning architectures to ingest language metrics.

#4 about 4 min

Binary and count vectorization techniques for text

Assuming document meaning directly from raw element frequency provides straightforward rule-based feature extraction.

#5 about 2 min

Word embeddings and dimensional representation of text

Grouping isolated terminology by spatial similarity captures underlying linguistic patterns in multi-dimensional vector space.

#6 about 3 min

Learning representations using Word2Vec skip-gram architectures

Predicting surrounding words across extensive training datasets automatically establishes context-aware numeric associations.

#7 about 6 min

Training a Word2Vec model with Python and Gensim

Preparing cleaned tokens and instantiating an embedding pipeline quickly establishes functional numerical feature matrices.

#8 about 3 min

Converting word embeddings into document-level vectors

Averaging individual element representations produces a single dense footprint encapsulating entire contextual phrases.

#9 about 2 min

Evaluating a basic clickbait classification neural network

Feeding averaged document matrices into dense classification heads rapidly yields high-accuracy predictive capabilities.

#10 about 4 min

Disambiguating text using transformer self-attention mechanisms

Adding positional scoring arrays allows architectural blocks to quantify exact relevance between distant contextual dependencies.

#11 about 4 min

Pre-training language architectures with the BERT model

Iteratively guessing masked parameters and tracking sentence flows develops profound internal linguistic intuition.

#12 about 2 min

Fine-tuning pre-trained pipelines for localized domains

Replacing default generalized inference heads focuses established neural connections onto specially targeted logic goals.

#13 about 6 min

Implementing tokenization using Hugging Face Transformers

Maintaining fixed tensor dimensions via padding offsets and boolean matrices prevents crippling inference disruptions.

#14 about 3 min

Training the DistilBERT sequence classification model

Running a fine-tuning optimizer pass seamlessly adapts powerful connection weights to solve custom categorization constraints.

#15 about 2 min

Selecting ideal text processing methods for complexity

Matching mathematical methods against precise computational budgets improves processing efficiency and prevents heavy over-engineering.

#16 about 14 min

Troubleshooting vector representations and data science pipelines

Addressing testing artifacts and mitigating missing dictionary items bolsters application stability in real-world environments.

Matching moments

2:23 min

Historical breakthroughs in natural language processing models

Mary Grygleski Mary Grygleski · LIVE

5:11 min

Defining GPT architectures and natural language processing tasks

Mary Grygleski Mary Grygleski · LIVE

1:52 min

Modern natural language processing using Transformers and deep learning

Jan Schweiger · World Congress 2022

3:52 min

Evolution of large language models and natural language processing

Natalie Pistunovich · LIVE

2:18 min

Major breakthroughs shaping the artificial intelligence landscape

Nico Axtmann · World Congress 2022

5:05 min

Exploring popular generative AI models and applications

Mary Grygleski Mary Grygleski · LIVE

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
10 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
17 days ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
19 days ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
24 days ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude thumbnail
6:16
26 days ago

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude

Chris Heilmann, Daniel Cranney

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas thumbnail
5:46
about 1 month ago

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas

Chris Heilmann, Daniel Cranney, Marc Plogas

Related videos

Hybrid AI: Next Generation Natural Language Processing thumbnail
21:01
World Congress 2022 • over 4 years ago

Hybrid AI: Next Generation Natural Language Processing

Jan Schweiger

Harry Potter and the Elastic Semantic Search thumbnail
57:52
LIVE • over 2 years ago

Harry Potter and the Elastic Semantic Search

Iulia Feroli

Enter the Brave New World of GenAI with Vector Search thumbnail
1:00:56
LIVE • over 2 years ago

Enter the Brave New World of GenAI with Vector Search

Mary Grygleski

Multimodal Generative AI Demystified thumbnail
56:55
LIVE • almost 3 years ago

Multimodal Generative AI Demystified

Ekaterina Sirazitdinova

What comes after ChatGPT? Vector Databases - the Simple and powerful future of ML? thumbnail
59:38
LIVE • almost 3 years ago

What comes after ChatGPT? Vector Databases - the Simple and powerful future of ML?

Erik Bamberg

Ranking Amazon Reviews by Quality with Pointwise Ratings learned from Pairwise Data thumbnail
47:26
LIVE • over 6 years ago

Ranking Amazon Reviews by Quality with Pointwise Ratings learned from Pairwise Data

Tanmay Bakshi

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.