Building a Semantic Cache for Agentic AI: Practical workshop on reducing cost & latency

Imagine your pager going off at 3 AM, only to find that your LLM costs are about to exceed their threshold. Generative AI applications become slow and expensive when every user request is routed directly to an LLM. This workshop introduces the fundamentals of semantic caching, explaining how it differs from traditional key-based caching. We’ll explore where semantic caching fits within a modern LLM stack and walk through practical implementation patterns. We’ll build using a real Semantic Cache for an AI Application, measure its effectiveness and improve its effectiveness.