> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI compatibility

Use the OpenAI SDKs with the models served on the B3OS node by changing the base URL; supported endpoints, extensions and differences.

`/v1/infer` speaks the OpenAI wire format, so the OpenAI SDKs work with the base URL changed:

| Where your code runs | Base URL | Key |
|---|---|---|
| Anywhere | `https://beta-api.b3os.org/v1/infer` | Your B3OS key |
| Inside a B3OS sandbox | `http://127.0.0.1:7700/v1/infer` | Not checked; pass any string |

Pick a chat model from the node's list (the list also holds the speech model, so filter on `b3os.kind`):

```bash
export B3OS_MODEL=$(curl -s https://beta-api.b3os.org/v1/infer/models \
  -H "Authorization: Bearer $B3OS_API_KEY" | jq -r '[.data[] | select(.b3os.kind == "chat")][0].id')
```

**TypeScript**

```ts
    import OpenAI from "openai";

    const client = new OpenAI({
      apiKey: process.env.B3OS_API_KEY,
      baseURL: "https://beta-api.b3os.org/v1/infer",
    });
    const completion = await client.chat.completions.create({
      model: process.env.B3OS_MODEL!,
      messages: [{ role: "user", content: "Name one river in Oregon." }],
    });
    console.log(completion.choices[0].message.content);
    ```
  **Python**

```python
    import os
    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["B3OS_API_KEY"],
        base_url="https://beta-api.b3os.org/v1/infer",
    )
    completion = client.chat.completions.create(
        model=os.environ["B3OS_MODEL"],
        messages=[{"role": "user", "content": "Name one river in Oregon."}],
    )
    print(completion.choices[0].message.content)
    ```
  **curl**

```bash
    curl -s https://beta-api.b3os.org/v1/infer/chat/completions \
      -H "Authorization: Bearer $B3OS_API_KEY" \
      -H "Content-Type: application/json" \
      -d "{\"model\": \"$B3OS_MODEL\", \"messages\": [{\"role\": \"user\", \"content\": \"Name one river in Oregon.\"}]}"
    ```

[Infer](/infer) describes the models the node serves.

## Endpoints

| Endpoint | Status |
|---|---|
| `POST /chat/completions` | Served: text and, on models with image input, images; tools and tool choice; `response_format` with JSON Schema; sampling parameters; streaming with `stream_options.include_usage` |
| `GET /models`, `GET /models/{model}` | Served, with a `b3os` object per model |
| `POST /audio/speech` | Served, plus the `timestamps: "word"` extension |
| `POST /embeddings` | Planned, for embedding models the node serves |

`GET /v1/infer/audio/voices`, which lists the voices of each speech model, is a B3OS endpoint and uses B3OS lists and [problem errors](/errors).

## Differences from OpenAI

- **No silent fallback.** A model the node does not serve fails with `model_not_available`. To send it to the B3IQ gateway instead, set `"route": "gateway"` in the request body. `Server-Timing` reports the route the request took as `infer_route`.
- **One choice.** `n` must be 1.
- **Unsupported parameters fail.** A parameter the model's runtime does not support returns `400` with `unsupported_parameter`; other parameters the runtime accepts pass through.
- **Extensions live in a `b3os` object.** Responses carry `b3os: {route, node}`, and each model in the list carries `b3os` with its kind, context length, quantization, support for tools, JSON Schema output and image input, the largest request it accepts, and your token and request limits for it. OpenAI SDKs ignore the field.
- **Word timestamps.** `POST /audio/speech` with `"timestamps": "word"` returns JSON with base64 audio and the start and end of each word, instead of audio bytes. In v0.1 speech is produced as `wav`, `pcm` or `mp3`; other formats return `unsupported_parameter` until enabled.
- **Errors** use OpenAI's shape, `{"error": {"message", "type", "code", "param"}}`, with the B3OS code in `error.code`. See [infer errors](/errors#infer).

Tokens are metered as `infer.input_tokens` and `infer.output_tokens` (characters as `infer.audio_chars` for speech) and are free in the beta, within each model's tokens-per-minute limit.

## Planned

An Anthropic Messages endpoint at `/v1/infer/messages` and E2B's sandbox API are planned for later releases.