Prompt Debt: Why Your AI System Is Frozen in Time

Every production issue leaves a trace. In traditional software, it’s often an another line of code. In AI systems, it’s often another sentence in the prompt. Over time, these sentences become difficult to maintain.

I saw this happen while building and running AI systems in production. A RAG-based agent that kept returning incorrect answers led to more prompt changes. MCP server tool descriptions become broader as new edge cases appeared. These instructions behaved differently with models and AI clients, hence more workarounds were implemented. Each change solved a problem, but together they made the system difficult to understand and maintain.

I call this prompt debt.

In this talk, I will explain how prompt debt builds up, why it often goes unnoticed until teams become afraid to change prompts or switch models, and why this is really a specification problem rather than a prompt engineering problem. We will start with a prompt for an AI application that works well. Then I will show how each fix adds another instruction until the prompt becomes difficult to understand and test. Finally, I will show how defining expected behavior through evaluation driven-specification makes it easier to compare models and iterate with confidence.

You will leave knowing how to identify prompt debt in your own systems, decide when a prompt needs refactoring instead of another workaround. You will also have a practical framework to prevent prompts from becoming the long-term specification of your application.