World Congress 2026 North America

Taming Rogue Agents: Observability-Driven Evaluation for Production Reliability

September 24, 2026 11:40 – 12:10 · 30 min Stage 4

What this session covers

Moving from a viral demo to reliable enterprise AI is the hardest hurdle in modern software engineering. Because agents are inherently non-deterministic, validating performance requires a granular look at the “why” and “how” behind every action.

This session explores how observability redefines agentic evaluation beyond binary pass/fail testing, shifting the focus from the result to the entire reasoning chain. Featuring a live demo using Arize Phoenix and LangGraph, we’ll showcase how to trace complex loops and automate evaluations in real-time.

What you’ll learn:

Beyond Output: Why judging final answers alone is a recipe for silent failure.

Live Implementation: Using Arize Phoenix with LangGraph to visualize traces and debug agentic “thought processes”.

The Playbook: An incremental framework for evaluation-first development.

Are you ready to stop guessing and start measuring? Join us to master the art of agent evaluation.

World Congress 2026 North America

Related talks

All sessions at this congress