> ## Documentation Index > Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI compatibility > Point an OpenAI SDK, or a framework built on one, at Streaming TTS and keep the code you have. Streaming TTS answers the request shape of the OpenAI speech API, `POST /audio/speech`, under the base URL `https://tts.us.assemblyai.com/v1/openai`. Point an OpenAI SDK, or a framework that is built on one, at that base URL with your AssemblyAI API key as its API key, and it speaks with AssemblyAI voices without other changes. This page is for code that already speaks the OpenAI speech API. For a new integration, the [Python and JavaScript SDKs](https://assemblyai.com/docs/tts/getting-started/quickstart.md) and the native [LiveKit](https://assemblyai.com/docs/tts/livekit.md) and [Pipecat](https://assemblyai.com/docs/tts/pipecat.md) plugins reach the whole API, word timings and barge-in on one session included. The route is part of the [HTTP API](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md) and behaves like [Create speech](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech.md): the audio streams as it is synthesized, the status waits for the first audio so every earlier failure is a real HTTP status, a failure after that point cuts the connection, and the same limits and billing apply. Only the request fields and the error body follow OpenAI; [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md) is the route reference. | Data zone | Base URL | | - | - | | US | `https://tts.us.assemblyai.com/v1/openai` | | EU | `https://tts.eu.assemblyai.com/v1/openai` | ## OpenAI SDKs > [tab] Python > > ```python theme={null} > import os > > import httpx2 > from openai import DefaultHttpx2Client, OpenAI > > client = OpenAI( > base_url="https://tts.us.assemblyai.com/v1/openai", > api_key=os.environ["ASSEMBLYAI_API_KEY"], > # The SDK's own pool sizes, with idle connections kept for 2 minutes > # instead of 5 seconds; see "Keep connections between replies" below. > http_client=DefaultHttpx2Client( > limits=httpx2.Limits( > max_connections=1000, max_keepalive_connections=100, keepalive_expiry=120 > ) > ), > ) > > with client.audio.speech.with_streaming_response.create( > model="assemblyai-tts-en", > voice="alloy", > input="Hello from AssemblyAI.", > response_format="pcm", > ) as response: > print("request id:", response.headers.get("x-request-id")) > response.stream_to_file("hello.pcm") > ``` > [tab] JavaScript > > ```javascript theme={null} > import fs from "node:fs/promises"; > import OpenAI from "openai"; > import { Agent, fetch } from "undici"; > > const client = new OpenAI({ > baseURL: "https://tts.us.assemblyai.com/v1/openai", > apiKey: process.env.ASSEMBLYAI_API_KEY, > // Idle connections kept for 2 minutes instead of Node's 4 seconds; see > // "Keep connections between replies" below. > fetch, > fetchOptions: { dispatcher: new Agent({ keepAliveTimeout: 120_000 }) }, > }); > > const response = await client.audio.speech.create({ > model: "assemblyai-tts-en", > voice: "alloy", > input: "Hello from AssemblyAI.", > response_format: "wav", > }); > await fs.writeFile("hello.wav", Buffer.from(await response.arrayBuffer())); > ``` The Python example is for `openai` 3, which is built on `httpx2`; with `openai` 2 or earlier, write `httpx` for `httpx2` and `DefaultHttpxClient` for `DefaultHttpx2Client`. The JavaScript example needs `npm install undici`, because the SDK takes a `dispatcher` only together with undici's own `fetch`; leave both out in a browser. The SDKs send the key as `Authorization: Bearer `, which the API accepts. `hello.pcm` holds raw 16-bit mono PCM at 24 kHz. In a browser, give the JavaScript SDK a [temporary token](https://assemblyai.com/docs/tts/authenticate-with-a-temporary-token.md) minted with `product=tts` as its `apiKey`, never your API key, and set its `dangerouslyAllowBrowser` option to `true`: without it the SDK refuses to run in a browser. ### Keep connections between replies Build the client once per process and reuse it. The API keeps an idle connection open for an hour, but the OpenAI SDKs close it much sooner: the Python SDK after 5 seconds, and the JavaScript SDK on Node after 4, the default of Node's `fetch`. A voice agent whose replies are further apart than that opens a new connection, a TCP and a TLS handshake, before nearly every reply. The `limits` and `dispatcher` settings in the examples above keep idle connections for 2 minutes. Every reconnect costs two network round trips before the request can leave: a few milliseconds close to the API, and more the farther your client is from it; see [Keep HTTP connections open](https://assemblyai.com/docs/tts/getting-started/optimizing-accuracy-and-latency.md#keep-http-connections-open). To open the connection before the first reply as well, call `client.models.list()` when a call starts; it reads [`GET /v1/openai/models`](https://assemblyai.com/docs/tts/api-reference/http-api/list-openai-models), which synthesizes nothing and needs no credential. To see where the wait goes, read the `Server-Timing` header of a `200`: it gives the server's part, and the rest of the time your client measures is the network and the connection setup; see [Keep latency low](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md#keep-latency-low). See [Optimizing accuracy and latency](https://assemblyai.com/docs/tts/getting-started/optimizing-accuracy-and-latency.md) for the other habits that shorten the wait. ## Voice agent frameworks LiveKit and Pipecat both have native AssemblyAI plugins, documented on [LiveKit](https://assemblyai.com/docs/tts/livekit.md) and [Pipecat](https://assemblyai.com/docs/tts/pipecat.md), which run on the WebSocket API with one session per call, barge-in without reconnecting and word-timed transcripts. Use those for a new agent. The OpenAI plugins below are for a pipeline already built on them, and give up the word timings. Voice agents should ask for `pcm`: raw PCM starts playing on the first bytes and needs no decoding, while MP3 adds about 46 ms of encoder silence (at 24 kHz) to the start of every sentence and its first byte arrives later; see [Stream raw PCM in the format your player needs](https://assemblyai.com/docs/tts/getting-started/optimizing-accuracy-and-latency.md#stream-raw-pcm-in-the-format-your-player-needs). > [tab] LiveKit > > ```python theme={null} > import os > > from livekit.plugins import openai > > tts = openai.TTS( > base_url="https://tts.us.assemblyai.com/v1/openai", > api_key=os.environ["ASSEMBLYAI_API_KEY"], > model="assemblyai-tts-en", > voice="alloy", > response_format="pcm", > ) > ``` > > The plugin already keeps idle connections for 2 minutes, and LiveKit opens one when the agent session starts, so it needs no change to keep connections between replies. > [tab] Pipecat > > ```python theme={null} > import os > > import httpx2 > from openai import DefaultAsyncHttpx2Client > from pipecat.services.openai.tts import OpenAITTSService > > tts = OpenAITTSService( > api_key=os.environ["ASSEMBLYAI_API_KEY"], > base_url="https://tts.us.assemblyai.com/v1/openai", > # Idle connections kept for 2 minutes instead of 5 seconds. > http_client=DefaultAsyncHttpx2Client( > limits=httpx2.Limits( > max_connections=1000, max_keepalive_connections=100, keepalive_expiry=120 > ) > ), > settings=OpenAITTSService.Settings(model="assemblyai-tts-en", voice="alloy"), > ) > ``` > > `http_client` keeps connections between replies, as in [OpenAI SDKs](#keep-connections-between-replies). Pipecat takes `http_client` from version 1.7, and `openai` 3 from 1.10; on 1.7 to 1.9, or with `openai` 2 or earlier, build it with `httpx` and `DefaultAsyncHttpxClient` instead. > > Pipecat's service always asks for `pcm` at 24 kHz, and accepts only OpenAI voice names, which the route maps to AssemblyAI voices (see [Voices](#voices)). ## Request fields `model` and `input` are required. `voice` names the voice, `response_format` the audio (`mp3` by default, `pcm` for a voice agent, `wav`, `mulaw` or `alaw`), and `stream_format: "sse"` asks for [speech events](#speech-events) instead of bytes. [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md) lists every field, its values and its refusals; the sections below cover the ones that are not plain OpenAI. ### Models | `model` | Speaks with | | - | - | | `assemblyai-tts-en`, `assemblyai-tts-es`, `assemblyai-tts-de`, `assemblyai-tts-it`, `assemblyai-tts-pt`, `assemblyai-tts-fr` | The default model of that language. The code also sets the language, so `assemblyai-tts-es` asks for Spanish | | A model's name, such as `assemblyai-tts-1` | That model, as `model_id` selects it on the [WebSocket API](https://assemblyai.com/docs/tts/api-reference/websocket-api.md). The name sets no language; see [Extensions](#extensions) for how the language is chosen | Names are case-insensitive. OpenAI's model names, such as `tts-1`, are refused with `400` and the code `model_not_found`; [List models (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/list-openai-models.md) answers the language model names, and the `X-TTS-Model` response header reports the model that spoke. ### Voices `voice` keeps its OpenAI name and takes one of these: * An OpenAI voice name, in any letter case. Each one speaks with an AssemblyAI voice in its place, as the table below shows. * The library slug of an [AssemblyAI voice](https://assemblyai.com/docs/tts/voices.md), such as `aai-jane`, in lowercase: the `voice_id` that names the voice on every other route. * A voice's ID, a UUID. A [cloned voice](#cloned-voices) can also go in as an object. A bare AssemblyAI voice name such as `jane` is not one of them. It is refused with `422` (`invalid_request_error`, code `invalid_parameter`, `param` `voice`), and the message names the slug to send, `aai-jane`. | OpenAI voice | AssemblyAI voice | | - | - | | `alloy` | `aai-jane` | | `ash` | `aai-michael` | | `ballad` | `aai-charles` | | `cedar` | `aai-george` | | `coral` | `aai-anna` | | `echo` | `aai-paul` | | `fable` | `aai-charles` | | `marin` | `aai-mary` | | `nova` | `aai-eve` | | `onyx` | `aai-george` | | `sage` | `aai-vera` | | `shimmer` | `aai-alba` | | `verse` | `aai-michael` | In English, an OpenAI voice name speaks with the AssemblyAI voice in this table. In any other language, every OpenAI voice name speaks with `aai-jane`, the default voice, though results are usually worse than in the voice's own language, English; to choose the voice of a Spanish or German request, send the slug or ID of a voice of that language instead. See [Voice and language](https://assemblyai.com/docs/tts/voices.md#voice-and-language). The response header `X-TTS-Voice-Id` gives the `voice_id` the request spoke with, and when you sent an OpenAI voice name, `X-TTS-Voice-Alias` holds it. Other values are not supported: a misspelt name is refused with `422` like a bare name, and a slug or UUID that names no voice with `400`, code `invalid_request`, both on `voice`. ### Cloned voices A [cloned voice](https://assemblyai.com/docs/tts/cloned-voices.md) is a voice you make in the AssemblyAI dashboard from a short reference recording of a speaker who consented to it. It belongs to your account, and it can be used for 60 days after its consent was last given unless you [renew the consent](https://assemblyai.com/docs/tts/cloned-voices.md#renew-consent). A cloned voice goes in `voice` as an object holding its ID or slug, `{"id": ""}`, which the OpenAI Python SDK passes through as it is, or as a string holding its UUID. The slug of one of your voices in a plain string is refused: send it as `{"id": ""}`. With a model's name, such as `assemblyai-tts-1`, the request speaks the voice's own language unless the [`language` extension](#extensions) names another; with a language's model, such as `assemblyai-tts-es`, it speaks that language. `assemblyai-tts-1` speaks a cloned voice in all six languages, though results are usually worse than in the voice's own language; see [Voice and language](https://assemblyai.com/docs/tts/voices.md#voice-and-language). ```python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://tts.us.assemblyai.com/v1/openai", api_key=os.environ["ASSEMBLYAI_API_KEY"], ) with client.audio.speech.with_streaming_response.create( model="assemblyai-tts-en", voice={"id": os.environ["ASSEMBLYAI_VOICE_ID"]}, input="Thanks for calling.", response_format="pcm", ) as response: print("voice_id:", response.headers.get("x-tts-voice-id")) response.stream_to_file("thanks.pcm") ``` The voice's refusals answer `400` `invalid_request_error` with the code `voice_not_found`, `voice_expired`, `voice_blocked`, `voice_not_paired`, `model_retired`, `model_not_found` or `not_owner`, and `biometric_terms_not_accepted` answers `403`; see [Errors](https://assemblyai.com/docs/tts/cloned-voices.md#errors). ### Extensions These fields are not part of the OpenAI request; most OpenAI clients pass them through an extra-body option (`extra_body` in the Python SDK): | Field | Values | | - | - | | `language` | A language name (`english`) or an ISO 639-1 code (`en`), in any letter case. Refused when it differs from the language `model` implies | | `sample_rate` | `8000`, `16000`, `22050`, `24000`, `44100`, `48000`; default `24000` | | `voice_id` | A voice's ID or slug, such as the slug of a [cloned voice](#cloned-voices), for clients that cannot send `voice` as an object. Leave `voice` out: next to any other `voice`, such as `alloy` or `aai-jane`, it is refused (`voice_id: conflicts with voice; give one of them`) | | `extend_biometric_consent` | `true` renews the biometric consent of a cloned voice named in `voice` as `{"id": ...}` or by its UUID, or in `voice_id`, and attests renewed consent on behalf of the account that owns it. Only the voice's owner may set it; see [Renew consent](https://assemblyai.com/docs/tts/cloned-voices.md#renew-consent) | The language is the one `language` sets, else the one the model name implies, else the language of the voice named by its slug or ID, else English. `assemblyai-tts-1` speaks every voice in all six languages, so an English voice can read Spanish text when the request asks for Spanish, though results are usually worse than in the voice's own language; see [Voice and language](https://assemblyai.com/docs/tts/voices.md#voice-and-language). Clients that cannot add fields pick a language with the model name. ```python theme={null} import os from openai import OpenAI client = OpenAI( base_url="https://tts.us.assemblyai.com/v1/openai", api_key=os.environ["ASSEMBLYAI_API_KEY"], ) with client.audio.speech.with_streaming_response.create( model="assemblyai-tts-en", voice="37e65973-9d96-5aab-adcd-01bf56ab5af9", input="Thanks for calling.", response_format="pcm", extra_body={"language": "en", "sample_rate": 16000}, ) as response: response.stream_to_file("thanks.pcm") ``` Word timings are not available on this route: `word_boundaries` must be left out or `false`. Use [Stream speech with word timings](https://assemblyai.com/docs/tts/api-reference/http-api/stream-speech-events.md) for them. ## Speech events With `"stream_format": "sse"` the response is `text/event-stream` in OpenAI's form: one `data:` line per event and no `event:` lines. While nothing is sent for 15 seconds, a `: keepalive` comment line arrives; skip lines that start with `:`. ```text theme={null} data: {"type":"speech.audio.delta","audio":"AAAAAP7/AQD+/wIA"} data: {"type":"speech.audio.delta","audio":"/f8DAP7/AQAAAP//"} data: {"type":"speech.audio.done","usage":{"input_tokens":0,"output_tokens":0,"total_tokens":0,"input_characters":22,"billed_characters":22,"audio_duration_seconds":1.62}} ``` * Each `speech.audio.delta` carries base64 of the next bytes of the `response_format`: with `wav`, the WAV header comes first; with `mp3`, whole MP3 frames. * `speech.audio.done` ends the stream. Its `usage` counts characters, not tokens: the token counts are always `0`, `input_characters` is the length of `input`, and `billed_characters` what the request is billed. There is no `[DONE]` line. * A failure after the first audio sends `{"type": "error", "error": {...}}`, with the fields of the error body below, and then cuts the connection. With `httpx`, read with `iter_bytes()` or `iter_lines()` and no `chunk_size`, or the `error` event can be lost with the cut; see [Streamed and buffered responses](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md#the-status-waits-for-the-first-audio). ## Errors Errors use OpenAI's body, so OpenAI SDKs raise their usual exceptions (`openai.BadRequestError`, `openai.RateLimitError`, ...). The `code` is the one the [HTTP API](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md#error-codes) uses: ```json theme={null} { "error": { "message": "model: 'tts-1' is not an AssemblyAI model; use one of GET /v1/openai/models, such as assemblyai-tts-en", "type": "invalid_request_error", "param": "model", "code": "model_not_found", "request_id": "93309b7e-9131-474f-99e4-6e0623463374", "retryable": false, "retry_after_seconds": null } } ``` Every status and code of the route is listed on [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md). In short: `400` for a bad field, model or voice; `401` for the credential; `402` for funds; `403` (`permission_error`) with `biometric_terms_not_accepted` when your account has not accepted the [biometric data terms](https://assemblyai.com/docs/tts/cloned-voices.md#biometric-data-terms) that its own cloned voices need; `422` for a `voice` string that is not an OpenAI name, a slug or an ID; `429` for both the requests-per-minute limit (`rate_limited`) and no room just now (`at_capacity`); and `503` when the service is unavailable. OpenAI SDKs retry a `429` on their own, twice by default. A request can be refused `at_capacity` after it was counted against your requests per minute, so each of those retries counts as a request too. To keep retries within your limit, set `max_retries` on the client and back off for `Retry-After` yourself. Every response carries the request's id in `X-Request-Id`, and a `200` also in `X-Generation-Id`; quote it in support requests. Send `X-Client-Request-Id` (1 to 512 printable ASCII characters) to have your own id echoed back. ## Related * [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md): the route reference, with every field, status and code. * [List models (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/list-openai-models.md) and [List models (OpenRouter)](https://assemblyai.com/docs/tts/api-reference/http-api/list-openrouter-models.md): the public models documents. * [HTTP API basics](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md): hosts, headers, limits and billing shared with the native routes; [Word boundaries](https://assemblyai.com/docs/tts/word-boundaries.md#over-the-http-api) for word timings over HTTP. * [LiveKit](https://assemblyai.com/docs/tts/livekit.md) and [Pipecat](https://assemblyai.com/docs/tts/pipecat.md): the native plugins. * [Streaming TTS on OpenRouter](https://assemblyai.com/docs/tts/openrouter.md): the same voices through OpenRouter's speech API, on an OpenRouter key. * [Optimizing accuracy and latency](https://assemblyai.com/docs/tts/getting-started/optimizing-accuracy-and-latency.md): client habits that shorten the time to first audio.