WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2026 Europe - Virtual Stage • Jun 30, 2026 • Session details

Load Testing AI: Aiming at a Moving Target

Heather Thacker

Are your traditional load tests hiding massive AI retry storms? Learn why LLM architectures require entirely new performance metrics to prevent violent latency cascades and runaway costs.

Pause
Mute Enter Fullscreen
#1 about 2 min

Why traditional API load testing fails for AI endpoints

Applying static web request metrics to AI systems obscures critical latency and cost issues in production.

#2 about 5 min

Mechanical differences between web APIs and AI traffic

AI endpoints introduce variable workload distributions, bounded GPU concurrency, and token-dependent generation costs.

#3 about 5 min

Five unique failure modes in production AI systems

Hidden queuing delays, token truncation, and retry storms introduce cascading performance degradation under heavy load.

#4 about 6 min

Modeling realistic AI workloads and user behavior testing

Using randomized prompt datasets and open workload models accurately simulates inference queuing and cache hit variability.

#5 about 3 min

Evaluating latency, cost, and resilience beyond request throughput

Tracking time to first token alongside generation limits transforms unpredictable billing into measurable performance guardrails.

#6 about 8 min

Contrasting static versus randomized prompt load testing outcomes

Replacing a single fixed string with a diverse prompt corpus reveals true tail latencies and processing times.

#7 about 6 min

Simulating AI system cascades and graceful load shedding

Pushing an endpoint beyond its inference capacity verifies whether bounded queues and circuit breakers prevent complete outages.

#8 about 5 min

Establishing AI-native service level objectives and safety guardrails

Implementing token-based performance targets and continuous cost observability ensures long-term operational resilience for AI services.

Matching moments

2:13 min

Identifying and hardening against generative AI risks

Rebekka Weiss Rebekka Weiss +1 · World Congress 2025

3:46 min

Integrating AI into web performance engineering workflows

Perf + AI

2:42 min

Managing AI token costs through strategic test design

Jakub Janczyk Jakub Janczyk · World Congress 2026 Europe

4:12 min

Measuring generative AI impact on team productivity

Justin Reock Justin Reock · World Congress 2026 Europe

54 sec

Architecting generative AI systems by targeted user load

Stan Girard Stan Girard · World Congress 2024

1:38 min

Scaling bottlenecks in generative AI applications

Stan Girard Stan Girard · World Congress 2024

More from World Congress 2026 Europe - Virtual Stage

One Does Not Simply Make Architecture Decisions Alone thumbnail
36:15
3 months ago

One Does Not Simply Make Architecture Decisions Alone

Vladas Diržys

From Black Box to Glass Box : Bedrock AgentCore Observability thumbnail
21:15
3 months ago

From Black Box to Glass Box : Bedrock AgentCore Observability

Yasemin Aktürk

Leading in Rhythm: How Women Make Better Decisions with Biological Intelligence thumbnail
30:41
3 months ago

Leading in Rhythm: How Women Make Better Decisions with Biological Intelligence

Julia Grosse

Better Together: Leveraging Your Observability Tools as a SIEM thumbnail
39:06
3 months ago

Better Together: Leveraging Your Observability Tools as a SIEM

Mathias Palmersheim

ChatGPT Apps: From Chat to Checkout thumbnail
30:29
3 months ago

ChatGPT Apps: From Chat to Checkout

Aaron Koivunen

Interior Decorating (TypeScript Decorators) thumbnail
29:22
3 months ago

Interior Decorating (TypeScript Decorators)

Michael Dowden

Related videos

Back to the Roots: Testing in the Age of AI thumbnail
29:50
World Congress 2026 Europe • 3 months ago

Back to the Roots: Testing in the Age of AI

Jakub Janczyk

Running AI at Scale: The Secret Ingredients thumbnail
30:04
World Congress 2026 Europe • 3 months ago

Running AI at Scale: The Secret Ingredients

Boris Hecker, Max Tschochohei, Peter Kürpick, Tomislav Tipurić

AI as a Test Designer: Transforming Experience into Automated Testing thumbnail
30:46
Europe 2026 Virtual • 3 months ago

AI as a Test Designer: Transforming Experience into Automated Testing

Alisa Hrustic

Why Most AI Features Fail After the Demo thumbnail
31:15
Europe 2026 Virtual • 3 months ago

Why Most AI Features Fail After the Demo

Akshay Nagpal

How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge thumbnail
42:26
Coffee With Developers • over 1 year ago

How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge

Meta Atamel, Guillaume Laforge

The Last Mile of AI: From Prototype to Production thumbnail
28:23
World Congress 2026 Europe • 3 months ago

The Last Mile of AI: From Prototype to Production

Clemens Wasner, Holger Hammel, Martin Remmelgas, Max Jacobson, Marius Cosareanu

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.