WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Oct 14, 2025

Minimal infrastructure for Real‑Time Phone Agents: transcripts in, responses out

Chris Heilmann , Daniel Cranney , Marius Obert , Staff Evangelist At Twilio

Skip the massive architectural hurdles of voice AI. Deploy real-time, context-aware phone agents in under 100 lines of TypeScript with Twilio Conversation Relay.

Pause
Mute Enter Fullscreen
#1 about 2 min

Minimal infrastructure for real-time phone agents

An overview of the minimal components required to build an AI agent accessible via standard phone calls.

#2 about 3 min

Comparing voice and text interaction speeds

Voice inputs offer a hands-free alternative while requiring consideration of cognitive load and synchronous processing limitations.

#3 about 3 min

Challenges of managing voice architecture and latency

Building a voice interface requires careful orchestration of speech recognition, interruption detection, and language model pipelines.

#4 about 3 min

Abstracting complexity with Twilio Conversation Relay

Twilio Conversation Relay manages speech-to-text, text-to-speech, and latency optimizations while decoupling core application logic.

#5 about 8 min

Building a WebSocket server with Deno

A Deno-based WebSocket server parses TwiML instructions, configuring voice models and handling initial connection headers.

#6 about 3 min

Connecting a Twilio phone number to the webhook

Purchasing a local phone number and linking its programmatic routing directly to the deployed web server.

#7 about 6 min

Integrating OpenAI for real-time streaming responses

Streaming large language model outputs minimizes latency by speaking words immediately as language chunks arrive rather than waiting for complete sentences.

#8 about 5 min

Maintaining conversation history and preserving context

Session memory stores historical conversation threads and maps unique request IDs across concurrent connection streams.

Matching moments

2:13 min

Building real-time phone agents with minimal infrastructure

Chris Heilmann +2 · LIVE

4:16 min

Addressing latency and architecture in voice agents

Chris Heilmann +2 · LIVE

1:10 min

Architecting a web real-time communication stack for agents

Nathaniel Okenwa Nathaniel Okenwa · WWC 2025

1:18 min

Introduction to real-time communication programming interfaces

Nathaniel Okenwa Nathaniel Okenwa · WWC 2025

8:23 min

Building a Deno server for Twilio integrations

Chris Heilmann +2 · LIVE

2:11 min

Analyzing voice interface research projects and technical limitations

Tobias Münch Tobias Münch · WWC 2024

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
5 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
5 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
12 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
19 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
26 days ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail
5:55
about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

From Software Agents to Physical Devices: Inside the Agentic Hardware Stack

Vivian Hu, Michael Yuan

Vivian Hu
Michael Yuan
Open session

World Congress 2026 North America

Designing APIs That Survive AI Agents at Scale

Phani Pendurthi

Mastercard, Principal Software Engineer

Phani Pendurthi
Open session

World Congress 2026 North America

No Single Model to Rule Them All: Building Resilient AI Agents Across Open & Closed LLMs

Emmanuel Acheampong

Senior Manager Developer Relations at Crusoe AI

Emmanuel Acheampong
Open session

World Congress 2026 North America

AI Agents are Only as Smart as their Context: Building a Real-Time Context Engine at Intuit

Bharat Patel

Lead Software Engineer at Intuit

Bharat Patel
Open session

World Congress 2026 North America

When Agents Became Users: Rearchitecting Identity and Permissions for AI at Scale

Dor Cohen, Yoav Gal

Dor Cohen
Yoav Gal
Open session

World Congress 2026 North America

Give the Agent a Budget, Not a Token

Sachin Malhotra

MTS @Anthropic

Sachin Malhotra

Related videos

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More thumbnail
1:16:15
LIVE • 10 months ago

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More

Chris Heilmann, Daniel Cranney, Marius Obert

Speech-to-Speech AI models - Marius Obert thumbnail
9:43
Coffee With Developers • about 2 months ago

Speech-to-Speech AI models - Marius Obert

Marius Obert

Hello JARVIS - Building Voice Interfaces for Your LLMS thumbnail
28:13
WWC 2025 • 12 months ago

Hello JARVIS - Building Voice Interfaces for Your LLMS

Nathaniel Okenwa

Performant Architecture for a Fast Gen AI User Experience thumbnail
25:47
WWC 2024 • almost 2 years ago

Performant Architecture for a Fast Gen AI User Experience

Nathaniel Okenwa

Raise your voice! thumbnail
30:59
LIVE • about 6 years ago

Raise your voice!

Lee Boonstra

Teaching APIs Beyond the Docs - Alex Goldman thumbnail
22:02
Coffee With Developers • 2 months ago

Teaching APIs Beyond the Docs - Alex Goldman

Alex Goldman

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.