Chat completions

POST /v1/infer/chat/completions is the OpenAI Chat Completions API. Set B3OS_MODEL to a chat model's ID from GET /v1/infer/models first (how).

A completion

bash
curl -s https://beta-api.b3os.org/v1/infer/chat/completions \  -H "Authorization: Bearer $B3OS_API_KEY" \  -H "Content-Type: application/json" \  -d "{\"model\": \"$B3OS_MODEL\", \"messages\": [{\"role\": \"user\", \"content\": \"Name one river in Oregon.\"}]}"

ts
import OpenAI from "openai";const client = new OpenAI({ apiKey: process.env.B3OS_API_KEY, baseURL: "https://beta-api.b3os.org/v1/infer" });const completion = await client.chat.completions.create({  model: process.env.B3OS_MODEL!,  messages: [{ role: "user", content: "Name one river in Oregon." }],});console.log(completion.choices[0].message.content, completion.usage);

python
import osfrom openai import OpenAIclient = OpenAI(api_key=os.environ["B3OS_API_KEY"], base_url="https://beta-api.b3os.org/v1/infer")completion = client.chat.completions.create(    model=os.environ["B3OS_MODEL"],    messages=[{"role": "user", "content": "Name one river in Oregon."}],)print(completion.choices[0].message.content, completion.usage)

text
Use the B3OS model to draft three subject lines for this release note: …

Tool calls

Pass tools (up to 128) and the model answers with tool_calls when it wants one; send the result back as a tool message:

json
{  "model": "b3os/your-chat-model",  "messages": [    { "role": "system", "content": "You are a concise assistant." },    { "role": "user", "content": "What is the weather in Eugene?" }  ],  "tools": [{    "type": "function",    "function": {      "name": "get_weather",      "description": "Current weather for a city.",      "parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] }    }  }]}

tool_choice takes none, auto, required or a named function, and parallel_tool_calls is accepted. The sandbox agent quickstart runs a full tool loop against the node.

JSON Schema output

response_format with type: "json_schema" constrains the answer to your schema, on models whose b3os.json_schema is true:

json
{  "model": "b3os/your-chat-model",  "response_format": {    "type": "json_schema",    "json_schema": {      "name": "city",      "schema": { "type": "object", "properties": { "name": { "type": "string" } }, "required": ["name"] }    }  },  "messages": [{ "role": "user", "content": "Name a city in Oregon as JSON." }]}

Streaming

With "stream": true the response is server-sent chat.completion.chunk objects ending with data: [DONE], exactly as OpenAI sends them. "stream_options": {"include_usage": true} adds a final chunk with the token counts. A stream may run up to 10 minutes.

Images

On models whose b3os.image_input is true, user messages can carry images, as in the OpenAI API.

Beyond the node's models

A model ID the node does not serve fails with model_not_available. Add "route": "gateway" to send the request to the B3IQ gateway instead. b3os.route in the response and infer_route in Server-Timing say which way it went.

When it fails

Errors use OpenAI's shape, with the B3OS code in error.code:

json
{ "error": { "message": "n must be 1 on this model", "type": "invalid_request_error", "param": "n", "code": "unsupported_parameter" } }