> Markdown version of [/events/world-congress-2026-europe/sessions/1341-testing-ai-agents](https://www.wearedevelopers.com/events/world-congress-2026-europe/sessions/1341-testing-ai-agents). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems


- **Date:** Friday, Jul 10, 2026
- **Time:** 14:20–14:50 (30 min)
- **Room:** Stage 6 - powered by Microsoft
- **Event:** World Congress 2026 Europe

## Recording

[Watch recording](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems)

## Description

AI Agents, chatbots, and RAG systems are easy to prototype — but difficult to test reliably. Small changes to prompts, models, retrieval sources, or system instructions can silently change behavior, and classic assertions (string matching, snapshots) often fail to capture what actually matters: correctness, relevance, grounded answers, and consistent multi-turn dialogue.

In this talk, we’ll start with the common testing problems in real projects: “it worked yesterday”, hidden regressions, evaluation noise, and the challenge of aligning developers and stakeholders on what “good” means.

Then we’ll explore practical testing possibilities with evaluation frameworks like DeepEval: how to validate responses beyond keyword matching, how to structure test cases for both chatbots and retrieval-based assistants, how to define pragmatic quality gates, and how to run these checks continuously in suggests, then?

As a side topic, we’ll show how BDD/Gherkin can wrap these evaluations into human-readable scenarios (Given–When–Then), making expectations reviewable by non-developers while keeping the actual validation powered by automated evaluation metrics.

You’ll leave with a reusable blueprint for introducing automated AI evaluation into your development workflow — from local runs to CI pipelines with actionable reports.

## Speaker

### [Sebastian Messingfeld](https://www.wearedevelopers.com/@sebastian-messingfeld)

Staff Engineer at Eurowings Digital


## Related talks at this congress

- [Beyond the Benchmark: How to Evaluate AI Agents in the Real World](https://www.wearedevelopers.com/events/world-congress-2026-europe/sessions/1311-beyond-the-benchmark) — Taylor Jordan Smith
- [The Art and Science behind evaluating AI Agents at scale](https://www.wearedevelopers.com/events/world-congress-2026-europe/sessions/961-the-art-and-science) — Alfonso Graziano
- [Are Classical Automation Frameworks Dead? How AI Agents Are Transforming QA](https://www.wearedevelopers.com/events/world-congress-2026-europe/sessions/1260-are-classical) — Andrei Nutas
- [Back to the Roots: Testing in the Age of AI](https://www.wearedevelopers.com/events/world-congress-2026-europe/sessions/1013-back-to-the-roots) — Jakub Janczyk