Max Tkacz

The AI Agent Path to Prod: Building for Reliability

Your AI agent works in demos, but will it break in production? Learn to build the evaluation frameworks and guardrails necessary for true reliability.

The AI Agent Path to Prod: Building for Reliability
#1about 4 minutes

Why AI agents fail in production environments

AI agents often fail in production because the probabilistic nature of LLMs conflicts with the need for reliability at scale.

#2about 5 minutes

Scoping an AI agent for a specific business problem

Start by identifying a low-risk, high-impact task, like automating free trial extensions, to establish a viable solution scope.

#3about 3 minutes

Walking through the naive V1 customer support agent

The initial agent uses an LLM with tools to fetch user data and extend trials, but its reliability is unknown without testing.

#4about 4 minutes

Using evaluations to test the happy path case

Evaluations are introduced as a testing framework to run the agent against specific test cases, revealing inconsistencies even in the happy path.

#5about 4 minutes

Improving agent consistency with prompt engineering

By adding explicit rules and few-shot examples to the system prompt, the agent's tool usage and response quality become more consistent.

#6about 5 minutes

Testing for prompt injection and other edge cases

A new evaluation case for prompt injection reveals a security flaw, which is fixed by adding specific security rules to the system prompt.

#7about 6 minutes

Applying production guardrails beyond evaluations

Beyond evals, production readiness requires adding human-in-the-loop processes, custom error handling, rate limiting, and model redundancy.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

The challenge of moving AI from demo to production
03:18 MIN

The challenge of moving AI from demo to production

What’s New with Google Gemini?

The gap between simple tutorials and production AI agents
02:15 MIN

The gap between simple tutorials and production AI agents

Building Agents Securely at Scale - Alfonso Graziano

Essential components for building production-ready agents
02:41 MIN

Essential components for building production-ready agents

Building Agents Securely at Scale - Alfonso Graziano

Moving agentic AI from proof of concept to production
02:05 MIN

Moving agentic AI from proof of concept to production

Building Blocks for Agentic Solutions in your Enterprise

Overcoming the challenges of productionizing AI models
04:51 MIN

Overcoming the challenges of productionizing AI models

Navigating the AI Revolution in Software Development

Key takeaways for building reliable LLM agents
01:06 MIN

Key takeaways for building reliable LLM agents

The Limits of Prompting: ArchitectingTrustworthy Coding Agents

Overcoming common adoption challenges with agentic AI
03:58 MIN

Overcoming common adoption challenges with agentic AI

Building and Modernising Apps with Agentic AI - Julia Kordick

The challenge of building production-ready AI applications
05:19 MIN

The challenge of building production-ready AI applications

One AI API to Power Them All

Featured Partners

Related Videos

See all videos
Agents for the Sake of Happiness
39:11

Agents for the Sake of Happiness

Thomas Dohmke

Beyond Chatbots: How to build Agentic AI systems
24:22

Beyond Chatbots: How to build Agentic AI systems

Philipp Schmid

On a Secret Mission: Developing AI Agents
29:03

On a Secret Mission: Developing AI Agents

Jörg Neumann

Beyond Prompting: Building Scalable AI with Multi-Agent Systems and MCP
25:32

Beyond Prompting: Building Scalable AI with Multi-Agent Systems and MCP

Viktoria Semaan

The Limits of Prompting: ArchitectingTrustworthy Coding Agents
24:05

The Limits of Prompting: ArchitectingTrustworthy Coding Agents

Nimrod Kor

Three years of putting LLMs into Software - Lessons learned
26:30

Three years of putting LLMs into Software - Lessons learned

Simon A.T. Jiménez

You are not my model anymore - understanding LLM model behavior
24:11

You are not my model anymore - understanding LLM model behavior

Andreas Erben

The State of GenAI & Machine Learning in 2025
26:47

The State of GenAI & Machine Learning in 2025

Alejandro Saucedo

Related Articles

View all articles
Daniel Cranney
Why Your AI Tool Fails After the Demo
AI tools often fail after the demo because organisations cannot operationalise them within existing production infrastructure. While early-stage pilots validate technical feasibility, long-term AI product adoption depends on integration clarity, infr...
Why Your AI Tool Fails After the Demo

From learning to earning

Jobs that call for the skills explored in this talk.