Topic mix
LLM evaluation
13 moments from 12 videos · 46:21 min total
Assess the quality and accuracy of your language model outputs. These curated talk segments cover automated benchmarking, human-in-the-loop workflows, and testing metrics.
Play section Defining and implementing LLM evaluation strategies
Defining and implementing LLM evaluation strategies
Because traditional unit tests fail on variable LLM outputs, utilizing a separate model for evaluation serves as an effective testing substitute.
Defining and implementing LLM evaluation strategies
Because traditional unit tests fail on variable LLM outputs, utilizing a separate model for evaluation serves as an effective testing substitute.
Play section The problem with single evaluation scores for LLMs
The problem with single evaluation scores for LLMs
Relying on a single evaluation for probabilistic models leads to flaky release decisions due to inconsistent responses.
The problem with single evaluation scores for LLMs
Relying on a single evaluation for probabilistic models leads to flaky release decisions due to inconsistent responses.
Play section Evaluating model performance and accuracy using LLM judges
Evaluating model performance and accuracy using LLM judges
Programmatic evaluation sets and LLM judges help objectively compare different models on accuracy, latency, and cost.
Evaluating model performance and accuracy using LLM judges
Programmatic evaluation sets and LLM judges help objectively compare different models on accuracy, latency, and cost.
Loading the rest of your playlist…
Your mix. Instantly.
More mixes
AI & Machine Learning
AI coding agents
AI & Machine Learning
MCP (Model Context Protocol)
AI & Machine Learning
RAG
Languages & Frameworks
TypeScript
Languages & Frameworks
React
Languages & Frameworks
Java
Cloud, Platform & Data
Platform engineering
Cloud, Platform & Data
Observability
Cloud, Platform & Data
Infrastructure as code
Architecture, Quality & Security
System architecture
Architecture, Quality & Security
Code review
Architecture, Quality & Security
Software supply chain security
Leadership, Career & Community
Engineering strategy
Leadership, Career & Community
Developer experience
Leadership, Career & Community