Beyond the Thor’s Hammer: Pragmatic Agentic AI with Caching, Reuse, and Cost Guardrails

Agentic AI has become the default “big hammer”: throw more agents at the workflow, add more tools, loop until it works. It feels magical—until the cost bell rings: token spend explodes, latency creeps up, and teams realize they’re repeatedly paying for the same reasoning across users, departments, and weeks.

This session is about bringing pragmatism without losing ambition. We’ll explore how to balance productivity and timely results by treating agent systems like any other production platform: measurable SLAs, predictable cost, and deliberate architecture. The centerpiece is a topic many teams overlook in the rush to “agentify” everything: cacheability.

We’ll break down what’s actually being transmitted between agents and users—prompts, retrieved context, tool results, intermediate plans, and final answers—and identify what’s repeated (often a lot) across org units. Then we’ll introduce practical caching patterns for agentic systems:

RAG and retrieval caching (query → top-k chunks, embeddings, rerank results)

tool-result caching (APIs/DB queries with TTL, idempotency keys, provenance)

response + reasoning artifact caching (answer reuse with “freshness” guards)

workflow memoization (agent step outputs keyed by inputs, policies, and versions)

organization-level knowledge reuse (shared “answer primitives” for FAQs, policies, and ideation)

But caching isn’t free: it creates complexity around staleness, governance, personalization, and security boundaries. We’ll cover how to design caches that are safe and evolvable: TTL + invalidation strategies, semantic cache keys, redaction/PII handling, per-tenant isolation, and observability for hit rate vs. correctness. You’ll leave with a set of architectural principles and an incremental roadmap to scale agentic AI sustainably—so the hammer stays useful even when costs matter.