WeAreDevelopers LIVE • Oct 14, 2025

Minimal infrastructure for Real‑Time Phone Agents: transcripts in, responses out

Chris Heilmann , Daniel Cranney , Marius Obert , Staff Evangelist At Twilio

Skip the massive architectural hurdles of voice AI. Deploy real-time, context-aware phone agents in under 100 lines of TypeScript with Twilio Conversation Relay.

Pause

Mute

Enter Fullscreen

#1 about 2 min

Minimal infrastructure for real-time phone agents

An overview of the minimal components required to build an AI agent accessible via standard phone calls.

#2 about 3 min

Comparing voice and text interaction speeds

Voice inputs offer a hands-free alternative while requiring consideration of cognitive load and synchronous processing limitations.

#3 about 3 min

Challenges of managing voice architecture and latency

Building a voice interface requires careful orchestration of speech recognition, interruption detection, and language model pipelines.

#4 about 3 min

Abstracting complexity with Twilio Conversation Relay

Twilio Conversation Relay manages speech-to-text, text-to-speech, and latency optimizations while decoupling core application logic.

#5 about 8 min

Building a WebSocket server with Deno

A Deno-based WebSocket server parses TwiML instructions, configuring voice models and handling initial connection headers.

#6 about 3 min

Connecting a Twilio phone number to the webhook

Purchasing a local phone number and linking its programmatic routing directly to the deployed web server.

#7 about 6 min

Integrating OpenAI for real-time streaming responses

Streaming large language model outputs minimizes latency by speaking words immediately as language chunks arrive rather than waiting for complete sentences.

#8 about 5 min

Maintaining conversation history and preserving context

Session memory stores historical conversation threads and maps unique request IDs across concurrent connection streams.

Matching moments

2:13 min

Building real-time phone agents with minimal infrastructure

Chris Heilmann +2 · LIVE

4:16 min

Addressing latency and architecture in voice agents

Chris Heilmann +2 · LIVE

1:10 min

Architecting a web real-time communication stack for agents

Nathaniel Okenwa

Nathaniel Okenwa · World Congress 2025

42 sec

Delivering dynamic voice interactions via ElevenLabs

Alexandra Mihai

Alexandra Mihai +2 · Europe 2026 Virtual

1:18 min

Introduction to real-time communication programming interfaces

Nathaniel Okenwa

Nathaniel Okenwa · World Congress 2025

8:23 min

Building a Deno server for Twilio integrations

Chris Heilmann +2 · LIVE

More from WeAreDevelopers LIVE

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude thumbnail

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude

Chris Heilmann, Daniel Cranney

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas thumbnail

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas

Chris Heilmann, Daniel Cranney, Marc Plogas

WeAreDevelopers - Of Agents and Robots thumbnail

WeAreDevelopers - Of Agents and Robots

Chris Heilmann, Daniel Cranney, Marc Plogas

WeAreDevelopers LIVE - How DevRel Makes Tech More Human thumbnail

WeAreDevelopers LIVE - How DevRel Makes Tech More Human

Chris Heilmann, Daniel Cranney

Fake or News: Smart Glasses in Court, Medical Advice Watches & Mushroom Electonics - Jason Torres thumbnail

Fake or News: Smart Glasses in Court, Medical Advice Watches & Mushroom Electonics - Jason Torres

Chris Heilmann, Daniel Cranney

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

World Congress 2026 North America

September 24, 2026 · 11:00–11:30

Stage 2

From Stateless to Stateful: Real-Time Voice & Messaging Agents with Twilio and AWS

Rishab Kumar

Staff Developer Evangelist @ Twilio

Rishab Kumar

World Congress 2026 North America

September 25, 2026 · 12:30–14:30

Stage 8

Build a Voice AI Agent with Twilio Agent Connect and Gemini Flash

Rishab Kumar

Staff Developer Evangelist @ Twilio

Rishab Kumar

World Congress 2026 North America

September 24, 2026 · 11:20–11:25

Outdoor Stage

Finding the Edges: Testing, Evaluating, and Monitoring Voice AI Agents Before Your Users Do

Matt Wyman

CEO at Okareo

Matt Wyman

World Congress 2026 North America

September 24, 2026 · 16:10–16:40

Stage 5

From Software Agents to Physical Devices: Inside the Agentic Hardware Stack

Michael Yuan, Vivian Hu

Michael Yuan

Vivian Hu

World Congress 2026 North America

September 24, 2026 · 13:30–14:00

Stage 5

The Pedagogy of AI: Agents, Agency, and a World Beyond Coding Tutorials

Alex Goldman, Andrew MacLean

Alex Goldman

Andrew MacLean

World Congress 2026 North America

September 24, 2026 · 14:10–14:40

Stage 1

Anatomy of an AI Request: Where Latency and Cost Are Really Born

Dan Fu

VP of Kernels at Together AI

Dan Fu

Related videos

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More thumbnail

LIVE • 11 months ago

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More

Chris Heilmann, Daniel Cranney, Marius Obert

Speech-to-Speech AI models - Marius Obert thumbnail

Coffee With Developers • 3 months ago

Speech-to-Speech AI models - Marius Obert

Marius Obert

Hello JARVIS - Building Voice Interfaces for Your LLMS thumbnail

World Congress 2025 • about 1 year ago

Hello JARVIS - Building Voice Interfaces for Your LLMS

Nathaniel Okenwa

Performant Architecture for a Fast Gen AI User Experience thumbnail

World Congress 2024 • about 2 years ago

Performant Architecture for a Fast Gen AI User Experience

Nathaniel Okenwa

Raise your voice! thumbnail

LIVE • about 6 years ago

Raise your voice!

Lee Boonstra

Teaching APIs Beyond the Docs - Alex Goldman thumbnail

Coffee With Developers • 3 months ago

Teaching APIs Beyond the Docs - Alex Goldman

Alex Goldman

Related articles

TP

Thomas Pamminger

Never delegate the understanding

Every year at WeAreDevelopers , hundreds of engineers share what they know on our stages. The recordings end up on YouTube, but the insights get lost: a 45-minute talk holds five minutes for you, f...

3 months ago

A finished orange Eames fiberglass armchair surrounded by blue blueprint sketches of the same chair.

CH

Chris Heilmann

WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?

Join Scott Hanselman at WWC24 to explore AI's role as a superhero or supervillain. Scott shares his 32 years of experience in software engineering, discusses AI myths, ethical dilemmas, and tech ad...

about 2 years ago

WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?

CH

Chris Heilmann

The Web We Broke (And Why AI Agents Are Paying the Price) - AgentCon Berlin

This is the accompanying post to the talk Chris Heilmann gave at AgentCon in Berlin on 19/05/2026, you can also see the slides and listen to it in this screencast: Thirty years of developer shortc...

4 months ago

Yahoo Pipes allowing you to remix HTML content

DC

Daniel Cranney

Clone Your Voice with ElevenLabs LLMs and Node

Recently we decided to see how easy it is to automate a video-generation pipeline, to see how an AI-generated video stacks up against one made by members of our team. As part of this, I (Dan Cranne...

3 months ago

A glowing audio wave floating in front of a blurry laptop

From learning to earning

Jobs that call for the skills explored in this talk.

Software Engineer - Video

Austin, TX, United States

Experienced $138k–203k Remote NoSQL Nagios Twilio

Machine Learning Engineer

Remote Spacy Twilio Pytorch

about 1 month ago

Twilio's next Senior Principal Field Architect - AI Agents

San Jose, CA, United States

Expert $284k Remote Twilio Chatbots Google Cloud

about 1 month ago

Software Engineer

Oakland, CA, United States

Expert $142k–208k Remote Redis NoSQL Twilio

Senior Software Engineer (L3)

San Francisco, CA, United States

Expert $141k–176k Remote Redis NoSQL Twilio

Machine Learning Engineer

Port Angeles, WA, United States

Expert $155k–194k Remote Docker Twilio Airflow