WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2026 Europe • Jul 9, 2026 • Session details

Your Tests Are Lying to You: Let AI Expose What You Missed

Maish Saidel-Keesing

Are your AI-generated tests creating a dangerous false sense of security? Discover how specification-driven prompting and mutation testing expose the critical business logic your automated suites missed.

Pause
Mute Enter Fullscreen
#1 about 3 min

The false sense of security in AI-generated test coverage

High test coverage metrics from AI models often miss critical business logic and unexpected edge cases.

#2 about 6 min

Five common failure patterns in AI-generated software tests

AI testing frequently fails through tautological assertions, API hallucinations, aggressive over-mocking, and generic assumptions.

#3 about 2 min

Prompting AI with specifications instead of application code

Feeding detailed business requirements into AI tools generates comprehensive test suites rather than simple generic assertions.

#4 about 4 min

Scaling test generation with cascading specification tiers

Stacking system, domain, and function-level rules creates reusable building blocks that automatically enforce comprehensive architectural standards.

#5 about 5 min

Essential checklist for reliable AI-generated test suites

A practical framework guides engineers through optimal mutation thresholds, edge-case probing questions, and appropriate integration testing boundaries.

#6 about 3 min

Evaluating AI test effectiveness using human judgment and traceability

Human oversight remains essential to ensure generated code maps directly to business requirements instead of functioning as mere coverage decoration.

#7 about 5 min

Implementing specification-driven AI testing in thirty minutes

A rapid workflow demonstrates writing functional requirements, prompting generative capabilities, and validating the output using mutation metrics.

Matching moments

3:45 min

Comparing human-led versus AI-led test ownership

Jakub Janczyk Jakub Janczyk · World Congress 2026 Europe

2:27 min

Using automated tests to validate AI generated codebases

YK Sugi YK Sugi · World Congress 2025

2:22 min

Why AI-generated unit tests fail to validate requirements

Maish Saidel-Keesing Maish Saidel-Keesing · World Congress 2026 Europe

3:37 min

Automating unit test generation to verify artificial intelligence output

Rainer Stropek Rainer Stropek · World Congress 2024

1:48 min

The future landscape of artificial intelligence in software testing

Alisa Hrustic Alisa Hrustic · Europe 2026 Virtual

2:12 min

The evolving role of software testing in AI development

Jakub Janczyk Jakub Janczyk · World Congress 2026 Europe

More from World Congress 2026 Europe

Building the Nervous System of AI - Michael Kagan (NVIDIA) thumbnail
29:17
3 months ago

Building the Nervous System of AI - Michael Kagan (NVIDIA)

Michael Kagan, Sead Ahmetović

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote thumbnail
19:41
3 months ago

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote

Christian Heilmann

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend thumbnail
32:06
3 months ago

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend

David vonThenen

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes thumbnail
35:13
3 months ago

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes

Irina Branovic

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬 thumbnail
24:23
3 months ago

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬

Jeremy Murray

Angstfreude - AI and Corporate Culture - The Thrill and the Threat thumbnail
30:26
3 months ago

Angstfreude - AI and Corporate Culture - The Thrill and the Threat

Axel Ebert, Karin Krobath

Upcoming sessions on this topic

Open session

AI Is the Easy Part: Why Faster Engineers Don't Make Faster Businesses

  • Bharat Sharma

    Booking

    Senior Engineering Manager

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Open session

Beware of Strangers Bearing Code: Open Source Trust in the Agent Era

  • Vikram Vaswani

    Consultant

Open session

What If AI Helps Us Build the Wrong Product Faster?

  • Sekhar Burra

    Sain Innovation

    CEO

Open session

Autopsy of an Autonomous Incident: When the Agent Made It Worse

  • Navin Pai

    StackGen

    Director of Engineering

Open session

Prompt Debt: Why Your AI System Is Frozen in Time

  • Suriya Prakash B

    Red Hat

    Principal Software Engineer

Related videos

Back to the Roots: Testing in the Age of AI thumbnail
29:50
World Congress 2026 Europe • 3 months ago

Back to the Roots: Testing in the Age of AI

Jakub Janczyk

AI as a Test Designer: Transforming Experience into Automated Testing thumbnail
30:46
Europe 2026 Virtual • 3 months ago

AI as a Test Designer: Transforming Experience into Automated Testing

Alisa Hrustic

Why Testing Matters in AI - Luise Freese and Elio Struyf thumbnail
28:29
Coffee With Developers • 7 months ago

Why Testing Matters in AI - Luise Freese and Elio Struyf

Luise Freese, Elio Struyf

How to Trust Code You Didn't Write thumbnail
29:42
World Congress 2026 North America • 14 days ago

How to Trust Code You Didn't Write

Jeff An, Itamar Friedman, Shubho Sengupta, Francesco Ciulla

How will artificial intelligence change the future of software testing? thumbnail
45:24
LIVE • almost 6 years ago

How will artificial intelligence change the future of software testing?

Evelyn Haslinger

Who Tests the AI? Building Trustworthy AI Systems at Enterprise Scale thumbnail
27:57
World Congress 2026 North America • 14 days ago

Who Tests the AI? Building Trustworthy AI Systems at Enterprise Scale

Him Raj Singh

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.