WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Coffee With Developers • Jun 17, 2026

Speech-to-Speech AI models - Marius Obert

Marius Obert

Marius Obert argues traditional transcription pipelines destroy essential conversational context. Discover how native speech-to-speech models preserve emotional nuances and deploy multimodal voice agents in under 70 lines of code.

Pause
Mute Enter Fullscreen
#1 about 2 min

Comparing transcription-based voice pipelines to native models

Routing audio through separate transcription and generation steps strips conversational nuances like hesitations.

#2 about 2 min

Advantages of using native speech-to-speech streaming models

Streaming audio directly through unified models improves latency and maintains consistent voices across languages.

#3 about 2 min

Loss of developer control in black box voice models

Unified voice models abstract away intermediary steps, making it difficult to inject custom logic or webhooks.

#4 about 2 min

Managing token costs and multimodal context in voice agents

Streaming cheaper text tokens alongside audio inputs allows developers to provide rich context efficiently.

#5 about 2 min

Detecting emotional intent directly from voice audio streams

Analyzing raw voice inputs enables models to identify subtle nuances like frustration that text transcriptions lose.

#6 about 2 min

Detecting answering machines and preventing automated voice fraud

Differentiating between human speakers and automated systems remains a complex challenge for continuous outbound calling.

#7 about 2 min

Integrating speech models using the Twilio real-time SDK

New tooling simplifies telephony integration by automatically handling codec translation and sample rate conversions.

Matching moments

7:39 min

Streaming text responses from an AI language model

Chris Heilmann +2 · LIVE

2:11 min

Analyzing voice interface research projects and technical limitations

Tobias Münch Tobias Münch · World Congress 2024

58 sec

Enabling live translations with AI voice-to-text pipelines

Alex Goldman Alex Goldman · Coffee With Developers

42 sec

Delivering dynamic voice interactions via ElevenLabs

Alexandra Mihai Alexandra Mihai +2 · Europe 2026 Virtual

1:44 min

Summarizing developer experience and artificial intelligence companions

Robert Hoffmann Robert Hoffmann +1 · LIVE

2:05 min

Challenges of managing voice architecture and latency

Chris Heilmann +3 · LIVE

More from Coffee With Developers

Using Agents to Curate the Content You See Online - Dan Gish thumbnail
12:47
5 days ago

Using Agents to Curate the Content You See Online - Dan Gish

Dan Gish

AI Vector Search at Scale - Ewa Szyszka - Ewa Szyszka thumbnail
11:14
5 days ago

AI Vector Search at Scale - Ewa Szyszka - Ewa Szyszka

Ewa Szyszka

What is Cross-Origin Storage - Thomas Steiner thumbnail
34:51
5 days ago

What is Cross-Origin Storage - Thomas Steiner

Thomas Steiner

Women in AI - Nalini Garg thumbnail
42:19
6 days ago

Women in AI - Nalini Garg

Nalini Garg

Agents and People Dynamics in Shared Chats - Sam Liu thumbnail
37:41
13 days ago

Agents and People Dynamics in Shared Chats - Sam Liu

Sam Liu

Why Token Reduction Isn’t Cost Reduction - Dave Anderson & Sarel Weinberger, PhD. thumbnail
41:09
19 days ago

Why Token Reduction Isn’t Cost Reduction - Dave Anderson & Sarel Weinberger, PhD.

Dave Anderson, Sarel Weinberger, Phd

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 24, 2026 · 11:00–11:30

Stage 4

From Voice Demo to Enterprise Production

Anahita Havewala, Anuj Gupta

Anahita Havewala
Anuj Gupta
Open session

World Congress 2026 North America

September 24, 2026 · 11:20–11:25

Outdoor Stage

Finding the Edges: Testing, Evaluating, and Monitoring Voice AI Agents Before Your Users Do

Matt Wyman

CEO at Okareo

Matt Wyman
Open session

World Congress 2026 North America

September 24, 2026 · 17:30–18:00

Stage 6

No Single Model to Rule Them All: Building Resilient AI Agents Across Open & Closed LLMs

Emmanuel Acheampong

Senior Manager Developer Relations at Crusoe AI

Emmanuel Acheampong
Open session

World Congress 2026 North America

September 24, 2026 · 11:00–11:30

Stage 2

From Stateless to Stateful: Real-Time Voice & Messaging Agents with Twilio and AWS

Rishab Kumar

Staff Developer Evangelist @ Twilio

Rishab Kumar
Open session

World Congress 2026 North America

September 25, 2026 · 09:00–09:30

Stage 7

AI Agents are Only as Smart as their Context: Building a Real-Time Context Engine at Intuit

Bharat Patel

Lead Software Engineer at Intuit

Bharat Patel
Open session

World Congress 2026 North America

September 25, 2026 · 10:20–10:50

Stage 5

Physical AI: 5 Things You Can Build That Aren't Another Chatbot

Vini Senger

Senior Technical Evangelist for Startups

Vini Senger

Related videos

Minimal infrastructure for Real‑Time Phone Agents: transcripts in, responses out thumbnail
29:35
LIVE • 11 months ago

Minimal infrastructure for Real‑Time Phone Agents: transcripts in, responses out

Chris Heilmann, Daniel Cranney, Marius Obert, Staff Evangelist At Twilio

Performant Architecture for a Fast Gen AI User Experience thumbnail
25:47
World Congress 2024 • about 2 years ago

Performant Architecture for a Fast Gen AI User Experience

Nathaniel Okenwa

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More thumbnail
1:16:15
LIVE • 11 months ago

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More

Chris Heilmann, Daniel Cranney, Marius Obert

Raise your voice! thumbnail
30:59
LIVE • about 6 years ago

Raise your voice!

Lee Boonstra

Hello JARVIS - Building Voice Interfaces for Your LLMS thumbnail
28:13
World Congress 2025 • about 1 year ago

Hello JARVIS - Building Voice Interfaces for Your LLMS

Nathaniel Okenwa

WeAreDevelopers LIVE - Local Transcriptions with Open-Source AI thumbnail
1:01:49
LIVE • 3 months ago

WeAreDevelopers LIVE - Local Transcriptions with Open-Source AI

Chris Heilmann, Daniel Cranney, Mattias Geniar

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.