Speechmatics Realtime and Agent STT

Overview

Best for: Long sessions (up to 48 h), broadcast and meeting captions with realtime diarization and large custom vocabularies; EU-hosted processing.

At a glance

Languages: vendor markets 50+, count not verified. Price is Realtime Standard (default); Enhanced $0.80/hr, Linden 1 agent model $0.30/hr. Concurrency is the Pro plan (Free 2). Session max 48 h. Realtime code-switching only via Melia 1, which is a preview. Latency tunable via max_delay (0.7 s in examples), no vendor claim.

Features

• partials (enable_partials)
• max_delay latency control
• realtime diarization (speaker and channel)
• speaker identification
• custom dictionary up to 20,000 items
• turn detection / force end of utterance
• translation add-on
• Agent STT with turn events and speaker-labelled segments

Pricing

Free tier: $100 in credits at sign-up, no card (pricing page).

Source: speechmatics.com

Setup

1. Create an API key in the Speechmatics portal.
2. Connect to wss://eu.rt.speechmatics.com/v2 (or global.rt.speechmatics.com/v2) with Authorization: Bearer <key>, or a short-lived JWT for browsers.
3. Send StartRecognition JSON with audio_format and transcription_config, wait for RecognitionStarted.
4. Send binary audio (AddAudio), read AddPartialTranscript / AddTranscript, finish with EndOfStream {last_seq_no}.
5. For voice agents use wss://global.rt.speechmatics.com/v2/agent (Agent STT).

Endpoint

wss://eu.rt.speechmatics.com/v2 ; wss://global.rt.speechmatics.com/v2 ; Agent STT: wss://global.rt.speechmatics.com/v2/agent

Authentication

Authorization: Bearer <API_KEY>; browser clients use a temporary JWT (createSpeechmaticsJWT with type rt and a ttl), passed as ?jwt=.

import asyncio, json, os, websockets  # pip install websockets>=14

URL = "wss://eu.rt.speechmatics.com/v2"
START = {"message": "StartRecognition",
         "audio_format": {"type": "raw", "encoding": "pcm_s16le", "sample_rate": 16000},
         "transcription_config": {"language": "en", "model": "enhanced",
                                  "enable_partials": True, "max_delay": 0.7}}

async def main():
    hdr = {"Authorization": f"Bearer {os.environ['SPEECHMATICS_API_KEY']}"}
    async with websockets.connect(URL, additional_headers=hdr) as ws:
        await ws.send(json.dumps(START))
        async def send():
            n = 0
            with open("audio_16k_mono.raw", "rb") as f:
                while chunk := f.read(3200):  # 100 ms
                    await ws.send(chunk); n += 1
                    await asyncio.sleep(0.1)
            await ws.send(json.dumps({"message": "EndOfStream", "last_seq_no": n}))
        async for msg in ws:
            d = json.loads(msg)
            if d["message"] == "RecognitionStarted":
                asyncio.create_task(send())
            elif d["message"] in ("AddPartialTranscript", "AddTranscript"):
                kind = "partial" if d["message"] == "AddPartialTranscript" else "FINAL"
                print(kind, d["metadata"]["transcript"])
            elif d["message"] in ("EndOfTranscript", "Error"):
                print(d); break

asyncio.run(main())

Written from the current docs. Check the vendor's SDK version before you ship.

Warnings

Plus 12 warnings that apply to all speech-to-text, live APIs. See category warnings.

Limits

• Concurrent realtime sessions: Free 2, Pro 50, Enterprise custom.
• Session ends at 48 hours, after 1 hour without AddAudio, or after 3 minutes with no audio or ping/pong.
• Custom dictionary over 20,000 items closes the socket with protocol_error.

Models and products

Docs and sources

Docs

• Pricing
• Models
• Realtime limits
• Realtime input formats
• Realtime API reference
• Agent STT

Sources used

• speechmatics.com/pricing
• docs.speechmatics.com/speech-to-text/models.md
• docs.speechmatics.com/speech-to-text/realtime/limits.md
• docs.speechmatics.com/speech-to-text/realtime/input.md
• docs.speechmatics.com/speech-to-text/agent-stt.md