OpenAI Realtime transcription sessions

Overview

Best for: Apps already on the OpenAI Realtime stack that want transcription in the same session model, or browser capture via WebRTC.

At a glance

Price is true delta streaming (gpt-live-transcribe or gpt-realtime-whisper, $0.017/min). gpt-transcribe is $0.27/hr but only returns text after a commit. gpt-live-transcribe has no server VAD; you commit turns yourself. Diarization model is file-only. Rate limits depend on account tier.

Features

• transcript deltas
• final transcript per committed turn
• client-side commit (input_audio_buffer.commit)
• prompt / keywords / language hints
• delay setting (minimal to xhigh) on gpt-live-transcribe
• WebRTC for browsers with ephemeral client secrets

Pricing

Free tier: None specific to transcription.

Source: developers.openai.com

Setup

1. Create an API key.
2. Server: open a Realtime WebSocket with Authorization: Bearer <key>; the OpenAI cookbook pattern uses wss://api.openai.com/v1/realtime?intent=transcription.
3. Send session.update with session.type='transcription', audio.input.format, transcription.model and turn_detection (null for gpt-live-transcribe).
4. Append base64 PCM with input_audio_buffer.append, commit each turn with input_audio_buffer.commit (your own VAD).
5. Browser: mint an ephemeral client secret on your server and connect via WebRTC.

Endpoint

wss://api.openai.com/v1/realtime?intent=transcription (route listed as v1/realtime/transcription_sessions on model pages)

Authentication

Authorization: Bearer <API_KEY> server side; ephemeral client secrets for browser WebRTC.

import WebSocket from "ws"; // npm i ws
import fs from "fs";

const ws = new WebSocket("wss://api.openai.com/v1/realtime?intent=transcription", {
  headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
});
ws.on("open", async () => {
  ws.send(JSON.stringify({ type: "session.update", session: { type: "transcription",
    audio: { input: { format: { type: "audio/pcm", rate: 24000 },
      transcription: { model: "gpt-live-transcribe" }, turn_detection: null } } } }));
  const pcm = fs.readFileSync("audio_24k_mono_s16le.raw");
  for (let i = 0; i < pcm.length; i += 4800) { // 100 ms at 24 kHz
    ws.send(JSON.stringify({ type: "input_audio_buffer.append",
      audio: pcm.subarray(i, i + 4800).toString("base64") }));
    await new Promise((r) => setTimeout(r, 100));
  }
  ws.send(JSON.stringify({ type: "input_audio_buffer.commit" })); // end of turn
});
ws.on("message", (data) => {
  const ev = JSON.parse(data);
  if (ev.type === "conversation.item.input_audio_transcription.delta") process.stdout.write(ev.delta);
  if (ev.type === "conversation.item.input_audio_transcription.completed") console.log("\nFINAL:", ev.transcript);
  if (ev.type === "error") console.error(ev.error);
});

Written from the current docs. Check the vendor's SDK version before you ship.

Warnings

Plus 12 warnings that apply to all speech-to-text, live APIs. See category warnings.

Limits

• Rate limits depend on account usage tier (not captured).
• Completion events from different turns can arrive out of order; match by item_id.

Models and products

Docs and sources

Docs

• Realtime transcription guide
• Pricing
• gpt-live-transcribe model
• gpt-realtime-whisper model
• gpt-transcribe model

Sources used

• developers.openai.com/api/docs/pricing
• developers.openai.com/api/docs/guides/realtime-transcription.md
• developers.openai.com/api/docs/models/gpt-live-transcribe.md
• developers.openai.com/api/docs/models/gpt-realtime-whisper.md
• developers.openai.com/api/docs/models/gpt-transcribe.md
• learn.microsoft.com/en-us/answers/a/12769224
• datacamp.com/tutorial/gpt-live-transcribe-api