WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Coffee With Developers • Jun 17, 2026

Speech-to-Speech AI models - Marius Obert

Marius Obert

Marius Obert argues traditional transcription pipelines destroy essential conversational context. Discover how native speech-to-speech models preserve emotional nuances and deploy multimodal voice agents in under 70 lines of code.

Pause
Mute Enter Fullscreen
#1 about 2 min

Comparing transcription-based voice pipelines to native models

Routing audio through separate transcription and generation steps strips conversational nuances like hesitations.

#2 about 2 min

Advantages of using native speech-to-speech streaming models

Streaming audio directly through unified models improves latency and maintains consistent voices across languages.

#3 about 2 min

Loss of developer control in black box voice models

Unified voice models abstract away intermediary steps, making it difficult to inject custom logic or webhooks.

#4 about 2 min

Managing token costs and multimodal context in voice agents

Streaming cheaper text tokens alongside audio inputs allows developers to provide rich context efficiently.

#5 about 2 min

Detecting emotional intent directly from voice audio streams

Analyzing raw voice inputs enables models to identify subtle nuances like frustration that text transcriptions lose.

#6 about 2 min

Detecting answering machines and preventing automated voice fraud

Differentiating between human speakers and automated systems remains a complex challenge for continuous outbound calling.

#7 about 2 min

Integrating speech models using the Twilio real-time SDK

New tooling simplifies telephony integration by automatically handling codec translation and sample rate conversions.

Matching moments

3:57 min

Advancements in synthetic speech and voice banking technology

Léonie Watson · A11y + AI

7:39 min

Streaming text responses from an AI language model

Chris Heilmann Chris Heilmann +2 · LIVE

2:11 min

Analyzing voice interface research projects and technical limitations

Tobias Münch Tobias Münch · World Congress 2024

58 sec

Enabling live translations with AI voice-to-text pipelines

Alex Goldman Alex Goldman · Coffee With Developers

49 sec

Voice cloning models and legacy programming languages

Andrew MacLean Andrew MacLean +2 · LIVE

42 sec

Delivering dynamic voice interactions via ElevenLabs

Alexandra Mihai Alexandra Mihai +2 · Europe 2026 Virtual

More from Coffee With Developers

Using Agents to Curate the Content You See Online - Dan Gish thumbnail
12:47
about 1 month ago

Using Agents to Curate the Content You See Online - Dan Gish

Dan Gish

AI Vector Search at Scale - Ewa Szyszka - Ewa Szyszka thumbnail
11:14
about 1 month ago

AI Vector Search at Scale - Ewa Szyszka - Ewa Szyszka

Ewa Szyszka

What is Cross-Origin Storage - Thomas Steiner thumbnail
34:51
about 1 month ago

What is Cross-Origin Storage - Thomas Steiner

Thomas Steiner

Women in AI - Nalini Garg thumbnail
42:19
about 1 month ago

Women in AI - Nalini Garg

Nalini Garg

Agents and People Dynamics in Shared Chats - Sam Liu thumbnail
37:41
about 1 month ago

Agents and People Dynamics in Shared Chats - Sam Liu

Sam Liu

Why Token Reduction Isn’t Cost Reduction - Dave Anderson & Sarel Weinberger, PhD. thumbnail
41:09
about 2 months ago

Why Token Reduction Isn’t Cost Reduction - Dave Anderson & Sarel Weinberger, PhD.

Dave Anderson, Sarel Weinberger, Phd

Related videos

Minimal infrastructure for Real‑Time Phone Agents: transcripts in, responses out thumbnail
29:35
LIVE • 12 months ago

Minimal infrastructure for Real‑Time Phone Agents: transcripts in, responses out

Chris Heilmann, Daniel Cranney, Marius Obert, Staff Evangelist At Twilio

Performant Architecture for a Fast Gen AI User Experience thumbnail
25:47
World Congress 2024 • about 2 years ago

Performant Architecture for a Fast Gen AI User Experience

Nathaniel Okenwa

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More thumbnail
1:16:15
LIVE • 12 months ago

WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More

Chris Heilmann, Daniel Cranney, Marius Obert

Raise your voice! thumbnail
30:59
LIVE • over 6 years ago

Raise your voice!

Lee Boonstra

Hello JARVIS - Building Voice Interfaces for Your LLMS thumbnail
28:13
World Congress 2025 • about 1 year ago

Hello JARVIS - Building Voice Interfaces for Your LLMS

Nathaniel Okenwa

WeAreDevelopers LIVE - Local Transcriptions with Open-Source AI thumbnail
1:01:49
LIVE • 4 months ago

WeAreDevelopers LIVE - Local Transcriptions with Open-Source AI

Chris Heilmann, Daniel Cranney, Mattias Geniar

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.