Agents Can't Iterate Against Tests That Lie
What this session covers
Coding agents write nearly all of your code. Now you need to prove it works.
Flaky end-to-end (E2E) tests in shared environments make that proof difficult and unreliable. With limited context, agents often reach for the wrong fix: increase the timeout or add a retry.
This talk is a practical case study in rebuilding trust in tests for AI-heavy engineering organizations. I’ll show the workflow that helped us reduce the share of PRs affected by E2E flakes from 100% to under 15% in six weeks, including the open-source libraries and agent skills we built to classify flaky tests, connect to observability signals, and decide which tests to keep, delete, or move down the pyramid.
Attendees will leave with a repeatable playbook for making coding agents safe to use at scale and for teaching them to root-cause failing tests rather than “fix” them with retries.
World Congress 2026 North America
Related talks
- Your Evals Passed. Your Agent Just Emptied a Database.
Tejas Pravinbhai Patel · Amazon
Fri 25 Sep · Stage 3
- Manufacturing trust: speed and safety in the age of agents


Mark Cavage & 2 more
Thu 24 Sep · Mainstage
- How AI Agents Tripled Our Test Coverage on a 1.8M-Line iOS Codebase
Kush Agrawal · Duolingo
Thu 24 Sep · Stage 1