xAI Grok Voice Agent API

Overview

Best for: Teams wanting simple flat per-minute pricing, built-in live web and X search, voice cloning, or an OpenAI-compatible fallback provider.

At a glance

Vendor claims sub-second latency (no ms figure). No language list for the voice agent (TTS 20, STT 38+). Compliance items are vendor claims. Tool calls and text inputs billed extra. Telephony via partners (Twilio, Voximplant).

Features

• server_vad turn detection with optional idle_timeout_ms re-engagement
• reasoning.effort high or none
• tools: web_search, x_search, file_search (Collections), remote MCP, custom functions
• custom cloned voices
• session resumption (opt-in conversation caching)
• replace (spoken substitutions before TTS), keyterms and language hints
• binary audio transport
• OpenAI Realtime client compatibility with a base URL change
• partner integrations: LiveKit, Pipecat, Twilio, Voximplant

Pricing

Free tier: None documented for the voice agent.

Source: docs.x.ai

Setup

1. Create an xAI account at console.x.ai, add billing credit and create an API key.
2. Server: connect to the realtime WebSocket with Authorization: Bearer <key> and send session.update.
3. Browser: your server calls POST https://api.x.ai/v1/realtime/client_secrets with {"expires_after": {"seconds": 300}} and returns the token; the browser opens the WebSocket with subprotocol xai-client-secret.<token>.
4. Stream input_audio_buffer.append and play response.output_audio.delta.
5. Pin grok-voice-think-fast-2.0 (or 1.0) instead of grok-voice-latest in production.

Endpoint

wss://api.x.ai/v1/realtime?model=grok-voice-latest

Authentication

API key as Authorization: Bearer on the server. Browsers use short-lived tokens from POST https://api.x.ai/v1/realtime/client_secrets passed in the Sec-WebSocket-Protocol header with the xai-client-secret. prefix.

import WebSocket from "ws"; // server side; browsers use an ephemeral token via the sec-websocket-protocol header
const ws = new WebSocket("wss://api.x.ai/v1/realtime?model=grok-voice-latest", {
  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
});
ws.on("open", () => ws.send(JSON.stringify({
  type: "session.update",
  session: {
    voice: "eve",
    instructions: "You are a friendly support agent. Keep answers short.",
    turn_detection: { type: "server_vad" },
    audio: {
      input: { format: { type: "audio/pcm", rate: 24000 } },
      output: { format: { type: "audio/pcm", rate: 24000 } },
    },
    tools: [{ type: "web_search" }],
  },
})));
export function sendAudio(pcm16) { // 24 kHz mono PCM16
  ws.send(JSON.stringify({ type: "input_audio_buffer.append", audio: pcm16.toString("base64") }));
}
ws.on("message", (raw) => {
  const ev = JSON.parse(raw.toString());
  if (ev.type === "response.output_audio.delta" || ev.type === "response.audio.delta")
    playPcm16(Buffer.from(ev.delta, "base64"));
  if (ev.type === "input_audio_buffer.speech_started") stopPlayback();
  if (ev.type === "error") console.error(ev.error);
});

Written from the current docs. Check the vendor's SDK version before you ship.

Warnings

Plus 14 warnings that apply to all voice-to-voice APIs. See category warnings.

Limits

• Session duration and concurrency limits are not published
• Resumption history dropped after 30 minutes of inactivity
• keyterms: max 100 terms, 50 characters each
• Ephemeral client secrets: expiry set by expires_after.seconds (example 300); session and anchor fields not supported

Models and products

Docs and sources

Docs

• Voice overview
• Voice agent API
• Ephemeral tokens
• Pricing

Sources used

• docs.x.ai/docs/guides/voice
• docs.x.ai/developers/model-capabilities/audio/voice-agent
• docs.x.ai/developers/model-capabilities/audio/ephemeral-tokens
• docs.x.ai/developers/models
• docs.x.ai/developers/pricing
• docs.x.ai/developers/models/grok-voice-think-fast-2.0
• eesel.ai/blog/grok-voice-think-fast-2-pricing