Tanmay Bakshi

What do language models really learn

Are language models truly creative, or just powerful mathematical optimizers? This talk reveals what LLMs actually learn beyond the hype.

What do language models really learn
#1about 7 minutes

The fundamental challenge of modeling natural language

Language models aim to create intuitive human-computer interfaces, but this is difficult because language syntax doesn't fully capture semantic meaning.

#2about 3 minutes

How deep learning models learn by transforming data

Deep learning works by performing a series of transformations on input data to warp its vector space until it becomes linearly separable.

#3about 3 minutes

Why the training objective is key to model behavior

The training objective, or incentive, dictates exactly what a model learns and can lead to unintended outcomes if not designed carefully.

#4about 8 minutes

From Word2Vec and LSTMs to modern transformers

The evolution from slow, non-contextual models like LSTMs to the parallel and deeply contextual transformer architecture solved major NLP challenges.

#5about 7 minutes

A practical demo of a character-level BERT model

A scaled-down, character-level transformer model demonstrates the 'fill in the blank' pre-training task by predicting masked characters in artist names.

#6about 2 minutes

What language models implicitly learn about language structure

By analyzing a model's internal weights, we can see it learns phonetic relationships and syntactic structures without ever being explicitly trained on them.

#7about 7 minutes

Why current generative models don't truly 'write'

Generative models like GPT are excellent at predicting the next word based on statistical patterns but lack the underlying thought process required for true, creative writing.

#8about 4 minutes

Exploring the future with Blank Language Models

Blank Language Models (BLM) offer a new training approach by filling in text in any order, forcing the model to consider both past and future context.

#9about 3 minutes

The need for better tooling to accelerate ML research

The complexity of implementing novel architectures like BLMs highlights the need for better infrastructure and compiled languages like Swift for TensorFlow to speed up innovation.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Shifting from talent acquisition to talent architecture
03:28 MIN

Shifting from talent acquisition to talent architecture

The Future of HR Lies in AND – Not in OR

How AI can create more human moments in HR
03:13 MIN

How AI can create more human moments in HR

The Future of HR Lies in AND – Not in OR

Understanding global differences in work culture and motivation
06:10 MIN

Understanding global differences in work culture and motivation

The Future of HR Lies in AND – Not in OR

Balancing business, technology, and people for holistic success
06:51 MIN

Balancing business, technology, and people for holistic success

The Future of HR Lies in AND – Not in OR

How the HR function has evolved over three decades
05:10 MIN

How the HR function has evolved over three decades

The Future of HR Lies in AND – Not in OR

Navigating ambiguity as a core HR competency
04:22 MIN

Navigating ambiguity as a core HR competency

The Future of HR Lies in AND – Not in OR

The importance of a fighting spirit to avoid complacency
06:04 MIN

The importance of a fighting spirit to avoid complacency

The Future of HR Lies in AND – Not in OR

Moving from 'or' to 'and' thinking in HR strategy
06:59 MIN

Moving from 'or' to 'and' thinking in HR strategy

The Future of HR Lies in AND – Not in OR

Featured Partners

Related Videos

See all videos
Multimodal Generative AI Demystified56:55

Multimodal Generative AI Demystified

Ekaterina Sirazitdinova

Creating Industry ready solutions with LLM Models58:00

Creating Industry ready solutions with LLM Models

Vijay Krishan Gupta & Gauravdeep Singh Lotey

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

A beginner’s guide to modern natural language processing56:46

A beginner’s guide to modern natural language processing

Jodie Burchell

Inside the Mind of an LLM27:11

Inside the Mind of an LLM

Emanuele Fabbiani

The pitfalls of Deep Learning - When Neural Networks are not the solution32:54

The pitfalls of Deep Learning - When Neural Networks are not the solution

Adrian Spataru & Bohdan Andrusyak

Lies, Damned Lies and Large Language Models29:40

Lies, Damned Lies and Large Language Models

Jodie Burchell

How AI Models Get Smarter35:16

How AI Models Get Smarter

Ankit Patel

Related Articles

View all articles
LM
Luis Minvielle
What Are Large Language Models?
Developers and writers can finally agree on one thing: Large Language Models, the subset of AIs that drive ChatGPT and its competitors, are stunning tech creations. Developers enjoying the likes of GitHub Copilot know the feeling: this new kind of te...
What Are Large Language Models?
CH
Chris Heilmann
With AIs wide open - WeAreDevelopers at All Things Open 2025
Last week our VP of Developer Relations, Chris Heilmann, flew to Raleigh, North Carolina to present at All Things Open . An excellent event he had spoken at a few times in the past and this being the “Lucky 13” edition, he didn’t hesitate to come and...
With AIs wide open - WeAreDevelopers at All Things Open 2025

From learning to earning

Jobs that call for the skills explored in this talk.