Speech

POST /v1/infer/audio/speech is the OpenAI speech API. It returns audio bytes, or, with the B3OS timestamps: "word" extension, JSON with the audio and the timing of each word. Render uses those timings to land clicks on the words that describe them and to cut captions.

Synthesize

bash
curl -s https://beta-api.b3os.org/v1/infer/audio/speech \  -H "Authorization: Bearer $B3OS_API_KEY" \  -H "Content-Type: application/json" \  -d '{"model": "kokoro-82m", "voice": "af_heart", "input": "Open the billing page.", "response_format": "mp3"}' \  -o speech.mp3

ts
import { writeFile } from "node:fs/promises";import OpenAI from "openai";const client = new OpenAI({ apiKey: process.env.B3OS_API_KEY, baseURL: "https://beta-api.b3os.org/v1/infer" });const speech = await client.audio.speech.create({  model: "kokoro-82m",  voice: "af_heart",  input: "Open the billing page.",  response_format: "mp3",});await writeFile("speech.mp3", Buffer.from(await speech.arrayBuffer()));

python
import osfrom openai import OpenAIclient = OpenAI(api_key=os.environ["B3OS_API_KEY"], base_url="https://beta-api.b3os.org/v1/infer")speech = client.audio.speech.create(    model="kokoro-82m",    voice="af_heart",    input="Open the billing page.",    response_format="mp3",)with open("speech.mp3", "wb") as f:    f.write(speech.content)

text
Record a narrated demo of the billing page.

The model and voice IDs above are examples: take real ones from GET /v1/infer/models (models whose b3os.kind is speech) and GET /v1/infer/audio/voices.

Word timestamps

bash
curl -s https://beta-api.b3os.org/v1/infer/audio/speech \  -H "Authorization: Bearer $B3OS_API_KEY" \  -H "Content-Type: application/json" \  -d '{"model": "kokoro-82m", "voice": "af_heart", "input": "Open the billing page.",       "response_format": "wav", "timestamps": "word"}'

json
{  "object": "speech",  "model": "kokoro-82m",  "voice": "af_heart",  "format": "wav",  "sample_rate": 24000,  "duration_ms": 1330,  "audio_base64": "UklGRiQAAABXQVZFZm10IBAAAAABAAEAwF0AAIC7AAACABAAZGF0YQAAAAA=",  "words": [    { "text": "Open", "start_ms": 40, "end_ms": 310 },    { "text": "the", "start_ms": 310, "end_ms": 420 },    { "text": "billing", "start_ms": 420, "end_ms": 820 },    { "text": "page.", "start_ms": 820, "end_ms": 1290 }  ]}

The values are illustrative. audio_base64 holds the audio in format, and each word has its start and end in milliseconds from the start of the audio.

Voices

GET /v1/infer/audio/voices lists each speech model's voices with their language and a link to a short sample. Filter with ?model=. It is a B3OS endpoint, so it returns a B3OS list and B3OS problem errors rather than OpenAI's shape. An unknown voice fails with voice_not_found.

Speech bills infer.audio_chars, the characters of input synthesized.