Azure OpenAI GPT Realtime API (Microsoft Foundry)

Overview

Best for: Enterprises already on Azure that need private networking, Entra ID, Microsoft contracts and US/EU Data Zone processing for OpenAI realtime voice.

At a glance

Global prices for gpt-realtime-2.1; Data Zone is 1.1x ($35.20/$70.40). Microsoft quotes transport latency only (WebRTC ~100 ms, WebSocket ~200 ms). Docs still state a 32k input token limit that conflicts with the 128k model context. Generic Azure free-account credits may apply.

Features

• function calling and remote MCP servers
• server_vad, semantic_vad or manual turn handling
• image input
• out-of-band responses
• Entra ID keyless auth and managed identity
• ephemeral tokens via /openai/v1/realtime/client_secrets
• SIP telephony
• Global, Data Zone (US/EU) deployment types

Pricing

Free tier: No free tier for realtime models (Azure free account credits can apply).

Source: prices.azure.com

Setup

1. Create an Azure subscription and a Microsoft Foundry resource in a supported region (East US 2 or Sweden Central are the documented realtime regions).
2. In the Foundry portal deploy a realtime model (e.g. gpt-realtime-2.1) as Global or Data Zone; note the deployment name.
3. Prefer Entra ID: assign Cognitive Services OpenAI User and request tokens for scope https://ai.azure.com/.default; otherwise copy the resource key.
4. Server or telephony: open the GA WebSocket (/openai/v1/realtime) with the deployment name as model. Do not add api-version.
5. Browser: your token service calls https://<resource>.openai.azure.com/openai/v1/realtime/client_secrets and the browser posts its SDP to https://<resource>.openai.azure.com/openai/v1/realtime/calls with the ephemeral token.
6. Request quota increases early: 100k TPM is small for realtime because of context re-billing.

Endpoint

wss://<resource>.openai.azure.com/openai/v1/realtime?model=<deployment> (WebSocket); https://<resource>.openai.azure.com/openai/v1/realtime/calls (WebRTC); https://<resource>.openai.azure.com/openai/v1/realtime/client_secrets (ephemeral tokens)

Authentication

api-key header or Authorization: Bearer <Entra ID token> on the server. Browsers get an ephemeral token from your token service; the more secure pattern proxies the SDP exchange so the browser never holds even the ephemeral token.

import WebSocket from "ws"; // server side; browsers use /openai/v1/realtime/client_secrets + WebRTC
const RESOURCE = process.env.AZURE_OPENAI_RESOURCE;      // e.g. my-foundry-resource
const DEPLOYMENT = process.env.AZURE_OPENAI_DEPLOYMENT;  // your deployment name, not the model name
const ws = new WebSocket(
  `wss://${RESOURCE}.openai.azure.com/openai/v1/realtime?model=${DEPLOYMENT}`,
  { headers: { "api-key": process.env.AZURE_OPENAI_API_KEY } } // or Authorization: Bearer <Entra token>
);
ws.on("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: {
      type: "realtime",
      instructions: "You are a friendly support agent. Keep answers short.",
      audio: {
        input: { format: { type: "audio/pcm", rate: 24000 }, turn_detection: { type: "server_vad" } },
        output: { format: { type: "audio/pcm", rate: 24000 }, voice: "marin" },
      },
    },
  }));
});
export function sendAudio(pcm16) { // 24 kHz mono PCM16, ~100 ms chunks
  ws.send(JSON.stringify({ type: "input_audio_buffer.append", audio: pcm16.toString("base64") }));
}
ws.on("message", (raw) => {
  const ev = JSON.parse(raw.toString());
  if (ev.type === "response.output_audio.delta") playPcm16(Buffer.from(ev.delta, "base64"));
  if (ev.type === "input_audio_buffer.speech_started") stopPlayback();
  if (ev.type === "error") console.error(ev.error);
});

Written from the current docs. Check the vendor's SDK version before you ship.

Warnings

Plus 14 warnings that apply to all voice-to-voice APIs. See category warnings.

Limits

• Max session duration 60 minutes (monitor expires_at in session.created)
• Default gpt-realtime quota in the Tier 1 table: 200 RPM and 100,000 TPM (GlobalStandard); higher tiers raise it (e.g. 300 RPM / 150,000 TPM)
• GPT-Live on Azure: concurrent sessions per subscription Default 10, Tier 1 25, Tier 2 50, Tier 3 200, Tier 4 300, Tier 5 500
• Realtime quota is separate from chat completions quota
• Doc still states 32,000 input / 4,096 output tokens for the Realtime API, which conflicts with the 128k model context on OpenAI

Models and products

Docs and sources

Docs

• GPT Realtime API how-to
• Realtime via WebRTC
• Realtime via WebSockets
• Quotas and limits
• Pricing

Sources used

• learn.microsoft.com/en-us/azure/foundry/openai/how-to/realtime-audio
• learn.microsoft.com/en-us/azure/foundry/openai/how-to/realtime-audio-webrtc
• learn.microsoft.com/en-us/azure/foundry/openai/how-to/realtime-audio-websockets
• learn.microsoft.com/en-us/azure/foundry/openai/quotas-limits
• prices.azure.com/api/retail/prices
• learn.microsoft.com/en-us/azure/ai-services/speech-service/voice-live