AssemblyAI Universal-3.6 Pro Realtime and Universal-Streaming

Overview

Best for: Voice agents and live captioning that want turn detection built in, multilingual code-switching, and US/EU data residency on a simple hourly price.

At a glance

Price and languages are for the default Universal-3.6 Pro model; Universal-Streaming is $0.15/hr (English, or 6 languages). Limit is 100+ new sessions per minute on paid, not a concurrent-stream cap. Diarization +$0.12/hr. Streaming PII pricing unconfirmed. HIPAA and SOC 2 are vendor claims, not re-verified.

Features

• partial and final turns (Turn messages with end_of_turn)
• turn detection with min_turn_silence / max_turn_silence / vad_threshold and latency modes
• keyterm prompting (up to 100 terms)
• general prompting (beta, Pro)
• streaming diarization and multichannel (paid)
• medical mode (paid)
• Voice Focus noise isolation (paid, Pro)
• PII redaction and profanity filter pages for streaming
• UpdateConfiguration mid-stream
• session-end webhooks
• US/EU data-zone endpoints

Pricing

Free tier: $50 free credit at signup, no card required.

Source: assemblyai.com

Setup

1. Get an API key from the dashboard.
2. Open wss://streaming.assemblyai.com/v3/ws?speech_model=universal-3-6-pro&sample_rate=16000 with header Authorization: <API_KEY>.
3. Stream 50-1000 ms binary chunks at real-time pace; read Begin, Turn and Termination messages.
4. Send {"type":"Terminate"} when the call ends, otherwise you keep paying.
5. For browsers, mint a temporary token server-side (expires_in_seconds, optional max_session_duration_seconds) and pass it as token= in the URL.

Endpoint

wss://streaming.assemblyai.com/v3/ws (US: wss://streaming.us.assemblyai.com/v3/ws, EU: wss://streaming.eu.assemblyai.com/v3/ws)

Authentication

Authorization header with the raw API key; browser clients use a temporary streaming token generated server-side.

import asyncio, json, os, websockets  # pip install websockets>=14
from urllib.parse import urlencode

params = {"speech_model": "universal-3-6-pro", "sample_rate": 16000}
URL = "wss://streaming.assemblyai.com/v3/ws?" + urlencode(params)

async def main():
    hdr = {"Authorization": os.environ["ASSEMBLYAI_API_KEY"]}
    async with websockets.connect(URL, additional_headers=hdr) as ws:
        async def send():
            with open("audio_16k_mono.raw", "rb") as f:  # 16-bit PCM mono
                while chunk := f.read(3200):  # 100 ms (must be 50-1000 ms)
                    await ws.send(chunk)
                    await asyncio.sleep(0.1)  # never faster than real time
            await ws.send(json.dumps({"type": "Terminate"}))  # stops billing
        asyncio.create_task(send())
        async for msg in ws:
            d = json.loads(msg)
            if d["type"] == "Turn":
                print("FINAL" if d.get("end_of_turn") else "partial", d["transcript"])
            elif d["type"] == "Termination":
                break

asyncio.run(main())

Written from the current docs. Check the vendor's SDK version before you ship.

Warnings

Plus 12 warnings that apply to all speech-to-text, live APIs. See category warnings.

Limits

• Rate limit is on NEW sessions per minute, not total open sessions: Free 5/min, paid 100+/min, auto-scales +10% when you use 70%+ of it.
• Max session 3 hours (error 3008); sessions not terminated auto-close after 3 hours and are billed for the full time.
• Audio chunk must be 50-1000 ms (error 3007); sending faster than real time is rejected (3007).
• Too many sessions gives close code 1008/3009.

Models and products

Docs and sources

Docs

• Pricing
• Model selection
• Endpoints and data zones
• Rate limits
• Errors and closures
• Turn detection

Sources used

• assemblyai.com/pricing
• assemblyai.com/docs/streaming/universal-streaming
• assemblyai.com/docs/streaming/endpoints-and-data-zones.md
• assemblyai.com/docs/streaming/rate-limits.md
• assemblyai.com/docs/streaming/common-session-errors-and-closures.md
• assemblyai.com/docs/streaming/turn-detection.md