> Markdown version of [/videos/2078-from-minutes-to-seconds-lessons-learned-optimizing-multi-turn-agentic-workflows](https://www.wearedevelopers.com/videos/2078-from-minutes-to-seconds-lessons-learned-optimizing-multi-turn-agentic-workflows). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# From minutes to seconds: Lessons learned optimizing multi-turn agentic workflows

AI workflow latency doesn't have to bottleneck your scaling. Discover architectural tweaks, from token masking to deterministic fallbacks, that slash processing time from minutes to seconds.

- **Speakers:** [Douglas Reiser](https://www.wearedevelopers.com/@douglas-reiser)
- **Event:** World Congress 2026 Europe - Virtual Stage
- **Published:** July 2, 2026
- **Duration:** 42:44
- **URL:** https://www.wearedevelopers.com/videos/2078-from-minutes-to-seconds-lessons-learned-optimizing-multi-turn-agentic-workflows

## Summary

AI workflows map directly to tokens, making latency and token cost the primary bottlenecks for scaling agentic systems. Using a multi-turn CRM data staging application as a baseline, developers face the challenge of processing unstructured notes into structured database entities without racking up enormous delays or API expenses. Optimizing these systems traces back to two fundamental levers: reducing total AI calls and generating fewer tokens per call to mitigate cumulative time-to-first-token delays.

Addressing token hygiene requires strategies like not parroting, which uses reference markers instead of asking the LLM to repeat input text, and replacing token-heavy identifiers with temporary placeholder masks before passing them to the model. Structurally, transitioning from a naive sequential data chain to a centralized workflow state object limits context bloat, allowing discrete tasks to fetch only necessary metadata. Developers must also optimize prompt prefix caching by pushing dynamic elements to the end of context windows. However, tracking metrics reveals that trimming input formats—such as converting JSON to text—can paradoxically trigger models to generate verbose, unwanted reasoning structures.

Ultimately, the most significant performance gains stem from architectural trade-offs, specifically replacing AI components with deterministic code for structured tasks like live database searches, or using LLMs strictly as a fallback for complex edge cases like relationship reconciliation. Dropping explicit reasoning parameters and switching to lighter models dramatically cuts costs, but evaluating these assumptions is critical to avoid degrading base capabilities. Because LLM responses are highly non-deterministic, marginal optimizations demand rigorous evaluation suites and comprehensive tracing through tools like Langfuse to ensure architectural tweaks represent statistical improvements rather than latency noise.

**Keywords:** agentic workflow optimization, token cost reduction, llm latency minimization, time to first token, tokens per second, prompt prefix caching, workflow state management, token hygiene strategies, deterministic fallback routing, crm entity extraction, llm context window optimization, reasoning token management, langfuse ai tracing, opentelemetry observability

## Chapters

1. **Introducing the multi-turn CRM agent scenario** (00:00) — An overview of a CRM AI workflow designed to extract and link entities from conversational text.
1. **Analyzing the data staging orchestration pipeline** (05:45) — How the orchestrator separates intent between sub-agents and deterministic workflow boundaries.
1. **Measuring latency and token usage via tracing** (08:53) — Tracking step-by-step LLM inputs and costs to establish baseline performance metrics.
1. **Calculating latency through time to first token** (10:45) — How model initialization times and token generation speeds dictate overall system latency.
1. **Reducing output tokens by referencing text** (13:01) — Using indexed markers to prevent large language models from unnecessarily repeating inputs.
1. **Trimming context schemas to control reasoning** (21:39) — How converting dense JSON inputs to plain text alters hidden model reasoning layers.
1. **Masking redundant data tokens with placeholders** (23:31) — Substituting lengthy IDs and URLs with reference keys to minimize context bloat.
1. **Maintaining prompt structures for prefix caching** (25:36) — Ordering dynamic variables last to maximize cache hit rates across multiple requests.
1. **Decoupling workflow states from conversational history** (28:10) — Managing function parameters directly from execution states rather than orchestrator transcripts.
1. **Replacing marginal AI calls with deterministic logic** (31:15) — Limiting language models to unpredictable edge cases while handling standardized steps in code.
1. **Toggling explainability features to lower inference footprint** (35:16) — Stripping debug reasoning tokens from production prompts to directly decrease generation delays.
1. **Benchmarking quality degradation on smaller models** (38:27) — Scaling inference engines down and accepting minor feature lapses for substantial budget relief.
1. **Reviewing cumulative token and latency optimizations** (39:56) — Comparing the aggregate baseline times against the optimized branch configurations.

## Related Moments

- [Optimizing AI infrastructure costs by maximizing token caching](https://www.wearedevelopers.com/videos/100024-what-500-production-environments-taught-us-about-shipping-ai-agents) (from "What 500+ Production Environments Taught Us About Shipping AI Agents")
- [Rebuilding complex agent architectures to optimize token utilization](https://www.wearedevelopers.com/videos/100010-ship-smarter-agents-not-bigger-prompts) (from "Ship Smarter Agents, Not Bigger Prompts")
- [Transitioning toward AI-first coding and managing token costs](https://www.wearedevelopers.com/videos/100198-the-last-mile-of-ai-from-prototype-to-production) (from "The Last Mile of AI: From Prototype to Production")
- [Optimizing AI token consumption and running local language models](https://www.wearedevelopers.com/videos/2135-wearedevelopers-live-streaming-html) (from "WeAreDevelopers LIVE - Streaming HTML")
- [Running agents efficiently at enterprise scale](https://www.wearedevelopers.com/videos/100132-the-agent-interface-layer-protocols-tools-and-trust-boundaries) (from "The Agent Interface Layer: Protocols, Tools and Trust Boundaries")
- [Addressing the gap between coding assistants and complex workflows](https://www.wearedevelopers.com/videos/100266-ai-won-t-fix-your-engineering-culture) (from "AI Won't Fix Your Engineering Culture")

## Related Articles

- [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [6 Open-Source Tools to Reduce Your Token Usage](https://www.wearedevelopers.com/magazine/746-6-open-source-tools-to-reduce-your-token-usage)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)

## Related Jobs

- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1355348-machine-learning-engineer) at **TWILIO**
- [Senior AI Agent Software Engineer (Go, Python) (m/f/x)](https://www.wearedevelopers.com/jobs/48277-senior-ai-agent-software-engineer-go-python-m-f-x) at **Dynatrace**
- [Principal Field Architect - AI Agents](https://www.wearedevelopers.com/jobs/ext/1442858-principal-field-architect-ai-agents) at **Twilio**
- [Staff Software Engineer, Copilot Experiences](https://www.wearedevelopers.com/jobs/ext/164361-staff-software-engineer-copilot-experiences) at **GitHub**
- [Staff, Machine Learning Engineer (L4)](https://www.wearedevelopers.com/jobs/ext/1202639-staff-machine-learning-engineer-l4) at **Twilio**