Twilio Media Streams
Raw call audio over WebSocket: bidirectional streams send you the caller's audio as base64 8 kHz mu-law and let you play audio back, so you can connect any STT/LLM/TTS or speech-to-speech model to a phone call.
Overview
Best for: Developers wiring their own pipeline or a speech-to-speech model (OpenAI Realtime, Gemini Live) to phone numbers.
At a glance
Raw 8 kHz mu-law audio transport; you handle barge-in and turn-taking. Voice minutes billed on top. Free tier is Twilio trial credit only. BAA not re-verified for Media Streams.
audio/x-mulaw 8000 Hz mono, base64 in JSON 'media' events
Same format sent back in 'media' events; 'clear' to flush on barge-in, 'mark' to track playback
Whatever your models support.
Not applicable (transport); your pipeline determines latency.
Twilio global.
Twilio offers BAAs for HIPAA-eligible products; not re-verified for Media Streams here.
Features
- telephony
- raw audio access
- barge-in via clear
- playback marks
- DTMF (bidirectional, inbound only)
- works with OpenAI/Gemini realtime APIs
Pricing
| What | Price | Unit |
|---|---|---|
| Media Streams | $0.0044 | per minute |
| Inbound local / outbound US | $0.0085 / $0.014 | per minute |
| SIP interface | $0.004 | per minute |
$0.0044 stream + $0.0085-$0.014 voice + your STT/LLM/TTS ($0.02-$0.10 depending on providers; speech-to-speech realtime models can exceed this).
Free tier: Twilio trial credit only.
Source: twilio.com
Setup
- Point a Twilio number's voice webhook at TwiML returning <Connect><Stream url="wss://..."/>.
- Accept the WebSocket; read 'start' for streamSid, decode 'media' payloads (mu-law 8 kHz) into your STT or realtime model.
- Send TTS audio back as mu-law 8 kHz base64 in 'media' events; send 'clear' when the caller interrupts.
Endpoint
wss:// endpoint you hostAuthentication
Twilio credentials for webhooks; validate X-Twilio-Signature
Quick start javascript
// Twilio bidirectional Media Stream skeleton
import express from "express";
import { WebSocketServer } from "ws";
const app = express();
app.post("/voice", (req, res) => res.type("text/xml").send(
'<Response><Connect><Stream url="wss://ai.example.com/media" /></Connect></Response>'));
const wss = new WebSocketServer({ server: app.listen(3000), path: "/media" });
wss.on("connection", (ws) => {
let streamSid;
ws.on("message", (raw) => {
const m = JSON.parse(raw);
if (m.event === "start") streamSid = m.start.streamSid;
if (m.event === "media") {
const ulaw8k = Buffer.from(m.media.payload, "base64");
sendToSTTorRealtimeModel(ulaw8k); // your pipeline
}
});
// when your TTS produces mu-law 8 kHz audio:
onAgentAudio((ulawChunk) => ws.send(JSON.stringify(
{ event: "media", streamSid, media: { payload: ulawChunk.toString("base64") } })));
onUserBargeIn(() => ws.send(JSON.stringify({ event: "clear", streamSid })));
});Written from the current docs. Check the vendor's SDK version before you ship.
Warnings
8 kHz mu-law in and out
You must convert to/from mu-law 8 kHz. Sending PCM16 or 24 kHz audio unconverted produces noise. Many realtime model APIs accept g711_ulaw directly, which avoids resampling.
You own barge-in
Without sending 'clear' on interruption, Twilio keeps playing buffered audio after the caller starts talking.
<Connect><Stream> blocks the rest of the TwiML
Nothing after it runs until the socket closes; plan transfers via the REST API or by closing the stream and redirecting.
Per-call track limits
Unidirectional streams, SIPREC, real-time transcription and AMD share a 4-track cap; exceeding it silently prevents the stream (stream-stopped callback).
Plus 12 warnings that apply to all platforms and telephony APIs. See category warnings.
Limits
- One bidirectional stream per call
- Unidirectional: max 4 tracks per call (shared)
- Stream resource API cannot start bidirectional streams
Models and products
| Name | Status |
|---|---|
| Bidirectional stream (<Connect><Stream>) | GA |
| Unidirectional stream (<Start><Stream>) | GA |
Docs and sources
Docs
Sources used
Message field names in the snippet come from long-standing Twilio docs (WebSocket messages page not fetched in this session).