Reinventing Testing Practices in the AI Era

AI-infused applications demand a rethinking of our testing practices. Developers face a new class of challenge as LLMs become standard integration points in modern applications: non-deterministic behavior that traditional testing approaches were never designed to handle. The current wave of distributed, orchestrated, agentic AI systems is evolving fast and, if we’re being honest, it smells a lot like the early days of microservices.

In this session, we’ll explore how your DevOps and testing practices must evolve when you wire AI into your applications. Not all AI failures look the same, and recognizing the difference is the first step toward building systems you can actually trust. We’ll walk through practical testing and observability strategies, using open source tools that give you confidence in AI-infused applications at every layer of the stack.

You’ll leave with a concrete mental model for reasoning about AI failures and one grounding question: What if AI was just an API call?