ElevenAgents (formerly Conversational AI / Agents Platform)

Overview

Best for: Customer-facing agents where voice quality and a managed builder (RAG, tools, telephony, evals) matter more than raw cost.

At a glance

Concurrency is the Starter plan (Free 4, Business 40, burst to 3x at $0.16/min). $0.08/min excludes LLM tokens. Free plan 15 min/month. BAA on Enterprise only. Docs also mention 70+ languages for the voice layer.

Features

• tool calling (server and client tools)
• MCP tools with approval
• RAG knowledge base
• proprietary turn-taking model
• interruptions and timeouts config
• workflow builder
• multimodal text+voice
• burst concurrency
• data residency endpoints

Pricing

Free tier: 15 call minutes per month, 4 concurrent calls.

Source: elevenlabs.io

Setup

1. Create an ElevenLabs account and build an agent in the dashboard (prompt, LLM, voice, tools, audio formats).
2. For public agents connect with ?agent_id; for private agents generate a signed URL server-side.
3. Stream base64 user_audio_chunk messages, play 'audio' events, answer 'ping' with 'pong'.
4. For phone calls attach a Twilio number or SIP trunk in the dashboard.

Endpoint

wss://api.elevenlabs.io/v1/convai/conversation?agent_id=<agent_id>

Authentication

agent_id query parameter for public agents; signed URL / token for private agents (xi-api-key for REST)

// npm i ws. Create the agent in the dashboard first; set its audio formats to pcm_16000.
// Public agents: agent_id in the URL. Private agents: mint a signed URL on your server.
import WebSocket from "ws";

const ws = new WebSocket(
  `wss://api.elevenlabs.io/v1/convai/conversation?agent_id=${process.env.ELEVEN_AGENT_ID}`
  // EU residency: wss://api.eu.residency.elevenlabs.io/...
);

ws.on("message", (raw) => {
  const ev = JSON.parse(raw.toString());
  switch (ev.type) {
    case "conversation_initiation_metadata":
      console.log(ev.conversation_initiation_metadata_event); // negotiated audio formats
      break;
    case "audio":
      play(Buffer.from(ev.audio_event.audio_base_64, "base64"));
      break;
    case "ping":
      ws.send(JSON.stringify({ type: "pong", event_id: ev.ping_event.event_id }));
      break;
  }
});

// 16 kHz mono 16-bit PCM from your mic
export function sendPcm(buf) {
  ws.send(JSON.stringify({ user_audio_chunk: buf.toString("base64") }));
}
function play(pcm) {}

Written from the current docs. Check the vendor's SDK version before you ship.

Warnings

Plus 14 warnings that apply to all voice-to-voice APIs. See category warnings.

Limits

• Concurrency 4 (Free) to 40 (Business) concurrent calls; burst to 3x at double price
• Some LLMs are unavailable when EU data residency is enabled
• Call time measured on connection duration, not speech time

Models and products

Docs and sources

Docs

• ElevenAgents overview
• WebSocket API
• LLM options and pass-through
• Pricing

Sources used

• elevenlabs.io/pricing/agents
• elevenlabs.io/docs/help-center/product/eleven-agents/how-much-does-eleven-agent...
• elevenlabs.io/blog/weve-lowered-api-agents-pricing-and-introduced-pay-as-you-go
• elevenlabs.io/docs/agents-platform/customization/llm
• elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket
• getmacha.com/blog/elevenlabs-agents-pricing-explained