World Congress 2026 Europe • Jul 10, 2026 • Session details

Beyond the Benchmark: How to Evaluate AI Agents in the Real World

Taylor Jordan Smith

Tyler Smith reveals that high benchmark scores mask fundamental AI execution failures. Ditch the happy-path demo and evaluate production agents using full-stack observability and continuous telemetry.

Starting at 14:49 · Watch from the beginning

Beyond the Benchmark: How to Evaluate AI Agents in the Real World thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

2:28 min

Introduction to building reliable AI agents in production

Max Tkacz

Max Tkacz · World Congress 2025

1:56 min

Introduction to evaluating AI code review agents

Sofia Rest

Sofia Rest · World Congress 2026 North America

2:34 min

Best practices for scaling AI agent experimentation

Sofia Rest

Sofia Rest · World Congress 2026 North America

2:40 min

Reviewing live performance of self-correcting AI engineering agents

Ingo Eichhorst

Ingo Eichhorst · World Congress 2026 Europe

2:43 min

Best practices for implementing reliable AI agent frameworks

Marcel Scherenberg

Marcel Scherenberg · World Congress 2025

1:00 min

Limitations of current performance benchmarks for agentic AI

Manuel Weichselbaum

Manuel Weichselbaum · World Congress 2026 Europe

More from World Congress 2026 Europe

Building the Nervous System of AI - Michael Kagan (NVIDIA) thumbnail

Building the Nervous System of AI - Michael Kagan (NVIDIA)

Michael Kagan +1

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote thumbnail

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote

Christian Heilmann · WeAreDevelopers

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend thumbnail

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend

David vonThenen · NetApp

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes thumbnail

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes

Irina Branovic · adjoe

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬 thumbnail

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬

Jeremy Murray · Stack8s

Angstfreude - AI and Corporate Culture - The Thrill and the Threat thumbnail

Angstfreude - AI and Corporate Culture - The Thrill and the Threat

Upcoming sessions on this topic

Did the Model Improve, or Just the Score? A Developer’s Guide to LLM Evaluation

Jack Fitzsimons

Blatent

CTO and Co-Founder

Teaching AI Coding Agents to Build It Right the First Time

Sajeetharan Sinnathurai

Microsoft

Principal Product Manager

The Developer’s Guide to the Agentic Git Forge: Agents That Remember

Haimantika Mitra

Entire

Senior Developer Advocate

Ship Happens: Guardrails for AI-Written Code

Anthony Nguyen

LaunchDarkly

Senior Solutions Engineer

Autopsy of an Autonomous Incident: When the Agent Made It Worse

Navin Pai

StackGen

Director of Engineering

From Agentic AI to Enterprise Ready: Building Agents That Scale, Secure and Deliver

Nagendra Bangalore

IBM

Director of Engineering

Related videos

What 500+ Production Environments Taught Us About Shipping AI Agents thumbnail

What 500+ Production Environments Taught Us About Shipping AI Agents

Liran Hason · Coralogix

World Congress Europe · 2026

The Missing Infrastructure for AI Agents thumbnail

The Missing Infrastructure for AI Agents

Chris Waterson · Guild.ai

World Congress North America · 2026

The AI Agent Path to Prod: Building for Reliability thumbnail

The AI Agent Path to Prod: Building for Reliability

Max Tkacz · n8n

World Congress · 2025

Taming Rogue Agents: Observability-Driven Evaluation for Production Reliability thumbnail

Taming Rogue Agents: Observability-Driven Evaluation for Production Reliability

Anagha Rumade +2

World Congress North America · 2026

Your Agents Need Observability Before They Need Better Models thumbnail

Your Agents Need Observability Before They Need Better Models

Julia Furst Morgado · Dash0

World Congress North America · 2026

The Next Wave of Agents: What’s Real, What’s Next, and What Matters thumbnail

The Next Wave of Agents: What’s Real, What’s Next, and What Matters

Mitesh Patel +3

World Congress North America · 2026

Related articles

JC

Jenna Charlton

DB

David Burns

How to Lead Quality Teams Through AI Transformation

If you scroll through your LinkedIn feed right now, you’d be forgiven for thinking the entire software industry has already been fully automated by Large Language Models (LLMs). But, for those of u...

6 days ago

Floating UI elements with a hand pointing to a plotted-point on a graph

EF

Elizabeth Fuentes Leone, AWS Developer Advocate, GenAI

From Prototype to Production: Build AI Agents with This Free 4-Course Learning Path

AI agents are moving from demos to production systems. However, most developers face challenges bridging this gap. This learning path shows you how. Interest in AI agents continues to grow in 2025....

12 months ago

Learn to Build Production-Ready AI Agents

AH

Aishi Huang

DC

Daniel Cranney

After You Ship AI: 5 Signals to Monitor in Production

Trustworthy AI isn’t a launch checkbox. It’s something you keep observing. A launch review can catch a lot of AI risks before users see them, but the launch is not the finish line, instead it is th...

12 days ago

An AI production monitoring dashboard

AH

Aishi Huang

CH

Chris Heilmann

Trustworthy AI Starts at Deployment: 5 Checks Before You Ship

Trustworthy AI is not a slogan. It is a release standard. Most AI features do not fail because nobody talked about ethics. They fail because they get shipped like ordinary features, even though the...

3 months ago

Am image showing a nebula inside a circle indicating a thinking AI.

From learning to earning

Jobs that call for the skills explored in this talk.

about 2 months ago • Verified

Principal Product Manager, Agentic Evals

San Jose, United States

Expert Remote Product Management

Applied AI Health Data System Engineer-Senior Manager

Detroit, MI, United States

Expert $124k–280k Git NoSQL Keras

Applied AI Health Data System Engineer-Senior Manager

Irvine, CA, United States

Expert $124k–280k Git NoSQL Keras

Applied AI Health Data System Engineer-Senior Manager

Austin, TX, United States

Expert $124k–280k Git NoSQL Keras

Applied AI Health Data System Engineer-Senior Manager

Des Moines, IA, United States

Expert $124k–280k Git NoSQL Keras

Applied AI Health Data System Engineer-Senior Manager

Los Angeles, CA, United States

Expert $124k–280k Git NoSQL Keras