OpenAI Agents SDK (voice / realtime agents)
Free open-source SDK (Python and TypeScript) with RealtimeAgent/RealtimeSession wrappers over the OpenAI Realtime API (browser WebRTC with ephemeral keys, server WebSocket, SIP), and a Python VoicePipeline for chained STT -> agent -> TTS.
Overview
Best for: Teams standardised on OpenAI models who want agent tooling/handoffs with realtime voice.
At a glance
Free SDK; cost is OpenAI model usage. SIP supported but you need a carrier for phone numbers.
Mic via WebRTC in browser; PCM/G.711 over WebSocket
Same
Per OpenAI realtime/transcription models.
Model-dependent.
OpenAI API regions.
Per OpenAI API terms (not covered here).
Features
- tools
- handoffs
- guardrails
- interruptions
- tracing
- SIP
Pricing
| What | Price | Unit |
|---|---|---|
| SDK | $0 | license |
Cost is the OpenAI model bill, priced in the realtime models segment.
Free tier: Open source.
Source: npmjs.com
Setup
- npm i @openai/agents (or pip install 'openai-agents[voice]').
- Server: mint a short-lived client key for the Realtime API; never ship your real API key to the browser.
- Browser: create a RealtimeAgent and RealtimeSession and connect with the ephemeral key (WebRTC, mic handled for you).
Endpoint
OpenAI Realtime APIAuthentication
Ephemeral client secret in browser; API key server-side
Quick start javascript
// Browser voice agent with the OpenAI Agents SDK (TypeScript/JS)
import { RealtimeAgent, RealtimeSession } from "@openai/agents/realtime";
const agent = new RealtimeAgent({
name: "Assistant",
instructions: "You are a friendly voice assistant. Keep answers short.",
});
const session = new RealtimeSession(agent);
// ephemeralKey comes from your server (OpenAI client secret endpoint)
const { ephemeralKey } = await fetch("/api/realtime-token").then(r => r.json());
await session.connect({ apiKey: ephemeralKey }); // WebRTC + mic in the browserWritten from the current docs. Check the vendor's SDK version before you ship.
Warnings
Ties you to OpenAI realtime pricing
The SDK is free but speech-to-speech realtime minutes are among the more expensive options; long sessions also grow context cost.
Never expose the API key
Browser sessions must use short-lived client secrets minted server-side.
Feature parity differs by language
VoicePipeline is documented for Python; check JS docs before relying on it.
No telephony numbers included
SIP support exists, but you still need a carrier (Twilio, Telnyx, etc.).
Plus 12 warnings that apply to all platforms and telephony APIs. See category warnings.
Limits
- OpenAI Realtime rate limits apply
Models and products
| Name | Status |
|---|---|
| RealtimeAgent / RealtimeSession (JS: @openai/agents/realtime) | GA |
| VoicePipeline (Python, openai-agents[voice]) | GA |
Docs and sources
Docs
Sources used
Current default realtime model name; whether JS has a chained VoicePipeline yet.