> Markdown version of [/videos/10-raise-your-voice](https://www.wearedevelopers.com/videos/10-raise-your-voice). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Raise your voice!

Ditch rigid wake words and consumer timeouts by building your own custom voice AI. Master the full architecture using Angular, Node.js, and Dialogflow to handle complex, enterprise-grade conversations.

- **Speakers:** Lee Boonstra
- **Event:** WeAreDevelopers LIVE
- **Published:** June 17, 2020
- **Duration:** 30:59
- **URL:** https://www.wearedevelopers.com/videos/10-raise-your-voice

## Summary

Building a custom voice AI assistant offers distinct advantages over consumer-grade ecosystems like Google Assistant or Alexa, specifically avoiding rigid microphone timeouts, consumer terms of service, and mandated wake words. Custom conversational AI provides tailored, touchless solutions for self-service airport kiosks, specialized hardware integrations, and contact center automation. These enterprise environments demand unique natural language understanding, as models must parse long-winded, complex user stories into actionable intents rather than relying on the brief, simple commands favored by consumer voice platforms.

Constructing this infrastructure involves bridging browser-based microphone capture with backend processing through bidirectional binary streams. A frontend Angular application uses WebRTC and the recordRTC library to overcome cross-browser audio limitations, capturing down-sampled chunks of stereo audio and transmitting them via Socket.io to a Node.js server. The backend orchestrates a pipeline of Google Cloud services, starting with the Speech-to-Text API to transcribe the streaming audio in real time. To elegantly support multilingual conversational capabilities without duplicating Dialogflow training models, developers can implement a translation layer that converts localized queries into a base language before intent matching occurs.

Once the audio is transcribed and translated, the text flows into Dialogflow to classify user intents, extract conversational entities, and query knowledge bases like imported airport FAQ documents. The resulting dialog is translated back to the native tongue and synthesized into highly realistic audio using WaveNet text-to-speech models. Delivering this audio buffer back to the frontend requires precise handling of HTML5 Web Audio API contexts, specifically to debug and bypass iOS security constraints that mandate active user gestures for sound playback. Finally, hosting the entire architecture on a platform like Google Cloud App Engine Flex ensures out-of-the-box HTTPS SSL compliance, which is a strict technical requirement for securely accessing client device microphones.

**Keywords:** custom conversational AI development, webrtc frontend audio capture, socket.io bidirectional streaming, dialogflow intent classification pipelines, google cloud speech-to-text API, wavenet text-to-speech synthesis, multilingual API translation layers, touchless self-service kiosk automation, contact center AI voice bots, html5 web audio API contexts, ios browser microphone permissions, node.js backend stream processing, app engine HTTPS deployment protocols, natural language understanding models

## Chapters

1. **Developing custom voice AI versus ecosystem platforms** (00:17) — Custom voice solutions enable specialized hardware and independent deployments without consumer platform restrictions.
1. **Voice AI architecture and interactive kiosk demo** (06:14) — The application processes frontend WebRTC streams through Google APIs to drive an interactive multilingual kiosk.
1. **Capturing browser microphone access with RecordRTC** (09:31) — The RecordRTC library normalizes cross-browser media quirks and establishes predictable stream sample rates.
1. **Streaming bi-directional audio via Socket.io** (13:47) — Socket.io-stream establishes realtime bidirectional endpoints to pass binary audio from Angular to Node.js.
1. **Converting continuous audio streams to text** (15:16) — The Google Speech-to-Text API requires strictly matched audio encoding and sample rates to process streamed chunks.
1. **Classifying conversational intents using Dialogflow models** (18:28) — Dialogflow applies natural language understanding to detect complex conversational intents, map entity parameters, and trigger automated webhook fulfillments.
1. **Reconciling multilingual voice inputs with Translate API** (22:16) — The dynamic translation client converts arbitrary foreign input back to baseline English to ensure accurate intent matching.
1. **Synthesizing human-like voice responses with Wavenet** (26:18) — The Text-to-Speech API transforms textual responses into playback-ready linear16 audio buffers.
1. **Playing synthesized backend audio objects in browsers** (27:49) — The Angular frontend parses returned audio strings into HTML5 AudioContext nodes to circumvent restrictive mobile playback rules.
1. **Securing application microphone access with HTTPS deployments** (29:34) — App Engine Flex provides the automated SSL certificates required by modern browsers to allow microphone recording endpoints.

## Related Moments

- [Analyzing voice interface research projects and technical limitations](https://www.wearedevelopers.com/videos/1142-is-the-web-ready-for-voice-user-interfaces) (from "Is the web ready for voice user interfaces?")
- [Building a personal assistant interface using web speech](https://www.wearedevelopers.com/videos/1142-is-the-web-ready-for-voice-user-interfaces) (from "Is the web ready for voice user interfaces?")
- [Summarizing developer experience and artificial intelligence companions](https://www.wearedevelopers.com/videos/884-forget-developer-platforms-think-developer-productivity) (from "Forget Developer Platforms, Think Developer Productivity!")
- [Architecting a web real-time communication stack for agents](https://www.wearedevelopers.com/videos/1641-hello-jarvis-building-voice-interfaces-for-your-llms) (from "Hello JARVIS - Building Voice Interfaces for Your LLMS")
- [Streaming text responses from an AI language model](https://www.wearedevelopers.com/videos/1729-wearedevelopers-live-real-time-phone-agents-unsafe-vpns-more) (from "WeAreDevelopers LIVE – Real-Time Phone Agents, Unsafe VPNs & More")
- [Enhancing conversational intent through modern large language models](https://www.wearedevelopers.com/videos/1641-hello-jarvis-building-voice-interfaces-for-your-llms) (from "Hello JARVIS - Building Voice Interfaces for Your LLMS")

## Related Articles

- [Clone Your Voice with ElevenLabs LLMs and Node](https://www.wearedevelopers.com/magazine/739-clone-your-voice-with-elevenlabs-llms-and-node)
- [WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?](https://www.wearedevelopers.com/magazine/469-wwc24-talk-scott-hanselman-ai-superhero-or-supervillain)
- [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix)
- [Dev Digest 138 - Are you secure about this?](https://www.wearedevelopers.com/magazine/486-dev-digest-138-are-you-secure-about-this)

## Related Jobs

- [Principal Field Architect - AI Agents](https://www.wearedevelopers.com/jobs/ext/1442858-principal-field-architect-ai-agents) at **Twilio**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [Staff Software Engineer, Copilot Experiences](https://www.wearedevelopers.com/jobs/ext/164361-staff-software-engineer-copilot-experiences) at **GitHub**
- [Senior Engineer, Infrastructure Platform](https://www.wearedevelopers.com/jobs/ext/328836-senior-engineer-infrastructure-platform) at **Intercom, Inc.**
- [Software Engineer (L4)](https://www.wearedevelopers.com/jobs/ext/1932007-software-engineer-l4) at **Twilio**