WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Oct 14, 2025

Minimal infrastructure for Real‑Time Phone Agents: transcripts in, responses out

Chris Heilmann , Daniel Cranney , Marius Obert , Staff Evangelist At Twilio

Skip the massive architectural hurdles of voice AI. Deploy real-time, context-aware phone agents in under 100 lines of TypeScript with Twilio Conversation Relay.

Pause
Mute Enter Fullscreen
#1 about 2 min

Minimal infrastructure for real-time phone agents

An overview of the minimal components required to build an AI agent accessible via standard phone calls.

#2 about 3 min

Comparing voice and text interaction speeds

Voice inputs offer a hands-free alternative while requiring consideration of cognitive load and synchronous processing limitations.

#3 about 3 min

Challenges of managing voice architecture and latency

Building a voice interface requires careful orchestration of speech recognition, interruption detection, and language model pipelines.

#4 about 3 min

Abstracting complexity with Twilio Conversation Relay

Twilio Conversation Relay manages speech-to-text, text-to-speech, and latency optimizations while decoupling core application logic.

#5 about 8 min

Building a WebSocket server with Deno

A Deno-based WebSocket server parses TwiML instructions, configuring voice models and handling initial connection headers.

#6 about 3 min

Connecting a Twilio phone number to the webhook

Purchasing a local phone number and linking its programmatic routing directly to the deployed web server.

#7 about 6 min

Integrating OpenAI for real-time streaming responses

Streaming large language model outputs minimizes latency by speaking words immediately as language chunks arrive rather than waiting for complete sentences.

#8 about 5 min

Maintaining conversation history and preserving context

Session memory stores historical conversation threads and maps unique request IDs across concurrent connection streams.

Matching moments

2:13 min

Building real-time phone agents with minimal infrastructure

Chris Heilmann Chris Heilmann +2 · LIVE

4:16 min

Addressing latency and architecture in voice agents

Chris Heilmann Chris Heilmann +2 · LIVE

1:10 min

Architecting a web real-time communication stack for agents

Nathaniel Okenwa Nathaniel Okenwa · World Congress 2025

6:38 min

Blending retro technology with conversational AI integrations

Andrew MacLean Andrew MacLean +2 · LIVE

42 sec

Delivering dynamic voice interactions via ElevenLabs

Alexandra Mihai Alexandra Mihai +2 · Europe 2026 Virtual

1:18 min

Introduction to real-time communication programming interfaces

Nathaniel Okenwa Nathaniel Okenwa · World Congress 2025

More from WeAreDevelopers LIVE

Fake or News - Claude's DNA Discovery, Quick Cloning & Crime on Insta thumbnail
6:07
3 days ago

Fake or News - Claude's DNA Discovery, Quick Cloning & Crime on Insta

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40
3 days ago

WeAreDevelopers LIVE - Playing in Production

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
17 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
24 days ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
26 days ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
about 1 month ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Related videos

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More thumbnail
1:16:15
LIVE • 12 months ago

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More

Chris Heilmann, Daniel Cranney, Marius Obert

Speech-to-Speech AI models - Marius Obert thumbnail
9:43
Coffee With Developers • 4 months ago

Speech-to-Speech AI models - Marius Obert

Marius Obert

Hello JARVIS - Building Voice Interfaces for Your LLMS thumbnail
28:13
World Congress 2025 • about 1 year ago

Hello JARVIS - Building Voice Interfaces for Your LLMS

Nathaniel Okenwa

Performant Architecture for a Fast Gen AI User Experience thumbnail
25:47
World Congress 2024 • about 2 years ago

Performant Architecture for a Fast Gen AI User Experience

Nathaniel Okenwa

Raise your voice! thumbnail
30:59
LIVE • over 6 years ago

Raise your voice!

Lee Boonstra

Teaching APIs Beyond the Docs - Alex Goldman thumbnail
22:02
Coffee With Developers • 4 months ago

Teaching APIs Beyond the Docs - Alex Goldman

Alex Goldman

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.