Create a chat completion

The OpenAI Chat Completions API: messages (text and, on models with `image_input`, images), tools and tool choice, `response_format` with JSON Schema, sampling parameters and streaming. With `stream: true` the response is server-sent events of `chat.completion.chunk` objects ending with `data: [DONE]`, exactly as OpenAI sends them; `stream_options.include_usage` adds a final usage chunk (and `"usage": null` on the others). A stream that fails after it started ends with an OpenAI error event (`data: {"error": {...}}`); one that fails before the first token is a plain HTTP error. The response head of a stream waits for the first generated token, so `Server-Timing` can carry `ttft`. Parameters this node does not take return `400 unsupported_parameter` naming the parameter. Besides the OpenAI parameters below, it takes the sampling extensions `top_k`, `min_p`, `repetition_penalty` and `logit_bias`, and `chat_template_kwargs.enable_thinking` (the model answers without a reasoning phase unless it is `true`; the reasoning then arrives in `reasoning_content`). `metadata`, `store` and `service_tier` are accepted and ignored. `null` means unset for every optional parameter. Image URLs are `https://` (fetched by the node through its egress guard, which refuses private, loopback and metadata addresses with `403 egress_denied`) or `data:image/{png,jpeg,webp,gif};base64,`. JSON Schemas (tool `parameters`, `response_format.json_schema.schema`) larger than 64 KiB or nested deeper than 32 levels are `400 unsupported_parameter`. Bills input and output tokens. Requests are admitted against your organization's per-minute limits on the model: one request, and the estimated prompt plus `max_completion_tokens` (or `max_tokens`, else 512 tokens), settled with the real counts when the request ends.

POST /v1/infer/chat/completions

Authentication

Bearer Token (b3os_sk_ API key or b3os_pat_ token) API Key (cookie: __Host-b3os_session)

Request Body required

application/json

model string REQUIRED

A model ID from `GET /v1/infer/models`.

messages object[] REQUIRED

Array of:

role string REQUIRED

Enum: system, developer, user, assistant, tool

content string | null | (object | object)[]

One of:

Option 1string | null

Option 2

Array of:

One of:

Option 1

type string REQUIRED

Enum: text

text string REQUIRED

Option 2

type string REQUIRED

Enum: image_url

image_url object REQUIRED

...

name string | null

tool_calls object[] | null

Array of:

id string REQUIRED

type string REQUIRED

Enum: function

function object REQUIRED

name string REQUIRED

arguments string REQUIRED

JSON-encoded arguments.

tool_call_id string | null

For `tool` messages, the call this is the result of.

refusal string | null

reasoning_content string | null

B3OS extension. The reasoning of an earlier assistant turn, passed back to the model's template.

tools object[] | null

Array of:

type string REQUIRED

Enum: function

function object REQUIRED

name string REQUIRED

description string

parameters object

A JSON Schema object (at most 64 KiB serialized and 32 levels deep), as OpenAI requires.

strict boolean

tool_choice string | null | object

One of:

Option 1string | null

Option 2

type string REQUIRED

Enum: function

function object REQUIRED

name string REQUIRED

parallel_tool_calls boolean | null

response_format object | null | object | object

One of:

Option 1

type string REQUIRED

Enum: text

Option 2

type string REQUIRED

Enum: json_object

Option 3

type string REQUIRED

Enum: json_schema

json_schema object REQUIRED

name string REQUIRED

description string

schema object

A JSON Schema object (at most 64 KiB serialized and 32 levels deep), as OpenAI requires.

strict boolean

temperature number | null

top_p number | null

Nucleus sampling. `0` keeps only the most probable token (passed to the runtime as 1e-9, the same thing, since vLLM takes values above 0 only).

max_tokens integer | null

max_completion_tokens integer | null

Wins over `max_tokens` when both are set.

stop string | null | string[]

One of:

Option 1string | null

Option 2

Array of: string

seed integer | null

n integer | null

Only 1 is supported.

presence_penalty number | null

frequency_penalty number | null

logprobs boolean | null

top_logprobs integer | null

user string | null

stream boolean | null

stream_options object | null

include_usage boolean | null

route string | null

B3OS extension. `gateway` sends the request to the B3IQ gateway when this node does not serve the model (models it serves stay local); without it an unknown model is `404 model_not_available`.

Enum: local, gateway, null

top_k integer | null

Runtime extension. Sample from the `top_k` most likely tokens; `-1` turns it off.

min_p number | null

Runtime extension. Minimum probability relative to the most likely token.

repetition_penalty number | null

Runtime extension. Values above 1 discourage repeating tokens.

logit_bias object | null

Token ID to bias added to its logit.

chat_template_kwargs object | null

Runtime extension. Only `enable_thinking` is taken.

enable_thinking boolean

Let the model reason before answering; the reasoning arrives in `reasoning_content`.

metadata object | null

Accepted and ignored.

store boolean | null

Accepted and ignored (nothing is stored).

service_tier string | null

Accepted and ignored.

Responses

200 The completion, or the chunk stream with `stream: true`.

application/json

id string REQUIRED

object string REQUIRED

Enum: chat.completion

created integer REQUIRED

Unix seconds.

model string REQUIRED

choices object[] REQUIRED

Array of:

index integer REQUIRED

message object REQUIRED

role string REQUIRED

Enum: assistant

content string | null REQUIRED

tool_calls object[]

Array of:

id string REQUIRED

type string REQUIRED

Enum: function

function object REQUIRED

name string REQUIRED

arguments string REQUIRED

JSON-encoded arguments.

refusal string | null

reasoning_content string

B3OS extension. The model's reasoning, when `chat_template_kwargs.enable_thinking` is on.

finish_reason string REQUIRED

Enum: stop, length, tool_calls, content_filter

logprobs object | null

usage object

prompt_tokens integer REQUIRED

completion_tokens integer REQUIRED

total_tokens integer REQUIRED

system_fingerprint string | null

b3os object

B3OS extension; OpenAI SDKs ignore it. On a stream it rides on the last chunk: the usage chunk with `stream_options.include_usage`, else the chunk with `finish_reason`.

route string

Enum: local, gateway

node string

timing object

Server-side milliseconds from the request head (`ttft_ms` on streams, `model_ms` without).

ttft_ms number

model_ms number

total_ms number

400 The request is invalid (`invalid_request`, `unsupported_parameter`, `context_length_exceeded`).

401 Missing or invalid credentials (`unauthorized`).

403 The key lacks `infer:run` or `infer:read` (`forbidden`), or the egress guard refused an image URL (`egress_denied`).

404 The model is not served on this node (`model_not_available`).

413 The request body exceeds the model's `max_request_bytes` (`payload_too_large`).

415 The body is not JSON (`unsupported_media_type`): send `Content-Type: application/json`.

429 A limit applies (`rate_limited` for requests, tokens or remote image bytes per minute, `concurrency_limited` for requests in flight for the organization, `quota_exceeded`; `credits_exhausted` arrives as 402).

500 Something failed inside B3OS (`internal`).

503 The model runtime, the voice service or the B3IQ gateway route is restarting, overloaded or not set up on this node (`infer_backend_unavailable`, `capacity_unavailable`), or the node is unavailable (`node_unavailable`).

curl -X POST 'https://beta-api.b3os.org/v1/infer/chat/completions' \  -H 'Authorization: Bearer YOUR_API_TOKEN' \  -H 'Content-Type: application/json' \  -d '{  "model": "string",  "messages": [    {      "role": "system",      "content": "string",      "name": "string",      "tool_calls": [        {          "id": "string",          "type": "function",          "function": {            "name": "string",            "arguments": "string"          }        }      ],      "tool_call_id": "string",      "refusal": "string",      "reasoning_content": "string"    }  ],  "tools": [    {      "type": "function",      "function": {        "name": "string",        "description": "string",        "parameters": {},        "strict": true      }    }  ],  "tool_choice": "none",  "parallel_tool_calls": true,  "response_format": {    "type": "text"  },  "temperature": 0,  "top_p": 0,  "max_tokens": 1,  "max_completion_tokens": 1,  "stop": "string",  "seed": 0,  "n": 1,  "presence_penalty": -2,  "frequency_penalty": -2,  "logprobs": true,  "top_logprobs": 0,  "user": "string",  "stream": false,  "stream_options": {    "include_usage": true  },  "route": "local",  "top_k": -1,  "min_p": 0,  "repetition_penalty": 0,  "logit_bias": {},  "chat_template_kwargs": {    "enable_thinking": true  },  "metadata": {},  "store": true,  "service_tier": "string"}'

const response = await fetch('https://beta-api.b3os.org/v1/infer/chat/completions', {  method: 'POST',  headers: {      "Authorization": "Bearer YOUR_API_TOKEN",      "Content-Type": "application/json"  },  body: JSON.stringify({    "model": "string",    "messages": [      {        "role": "system",        "content": "string",        "name": "string",        "tool_calls": [          {            "id": "string",            "type": "function",            "function": {              "name": "string",              "arguments": "string"            }          }        ],        "tool_call_id": "string",        "refusal": "string",        "reasoning_content": "string"      }    ],    "tools": [      {        "type": "function",        "function": {          "name": "string",          "description": "string",          "parameters": {},          "strict": true        }      }    ],    "tool_choice": "none",    "parallel_tool_calls": true,    "response_format": {      "type": "text"    },    "temperature": 0,    "top_p": 0,    "max_tokens": 1,    "max_completion_tokens": 1,    "stop": "string",    "seed": 0,    "n": 1,    "presence_penalty": -2,    "frequency_penalty": -2,    "logprobs": true,    "top_logprobs": 0,    "user": "string",    "stream": false,    "stream_options": {      "include_usage": true    },    "route": "local",    "top_k": -1,    "min_p": 0,    "repetition_penalty": 0,    "logit_bias": {},    "chat_template_kwargs": {      "enable_thinking": true    },    "metadata": {},    "store": true,    "service_tier": "string"  })});const data = await response.json();console.log(data);

import requestsheaders = {    'Authorization': 'Bearer YOUR_API_TOKEN'}response = requests.post('https://beta-api.b3os.org/v1/infer/chat/completions', headers=headers, json={  "model": "string",  "messages": [    {      "role": "system",      "content": "string",      "name": "string",      "tool_calls": [        {          "id": "string",          "type": "function",          "function": {            "name": "string",            "arguments": "string"          }        }      ],      "tool_call_id": "string",      "refusal": "string",      "reasoning_content": "string"    }  ],  "tools": [    {      "type": "function",      "function": {        "name": "string",        "description": "string",        "parameters": {},        "strict": true      }    }  ],  "tool_choice": "none",  "parallel_tool_calls": true,  "response_format": {    "type": "text"  },  "temperature": 0,  "top_p": 0,  "max_tokens": 1,  "max_completion_tokens": 1,  "stop": "string",  "seed": 0,  "n": 1,  "presence_penalty": -2,  "frequency_penalty": -2,  "logprobs": true,  "top_logprobs": 0,  "user": "string",  "stream": false,  "stream_options": {    "include_usage": true  },  "route": "local",  "top_k": -1,  "min_p": 0,  "repetition_penalty": 0,  "logit_bias": {},  "chat_template_kwargs": {    "enable_thinking": true  },  "metadata": {},  "store": true,  "service_tier": "string"})print(response.json())

200 Response

{  "id": "chatcmpl-01m5t5tmscqev5tf32q87tr1v5",  "object": "chat.completion",  "created": 1793001600,  "model": "b3os/qwen3.5-9b",  "choices": [    {      "index": 0,      "message": {        "role": "assistant",        "content": null,        "tool_calls": [          {            "id": "call_7f3a",            "type": "function",            "function": {              "name": "get_weather",              "arguments": "{\"city\":\"Eugene\"}"            }          }        ]      },      "finish_reason": "tool_calls",      "logprobs": null    }  ],  "usage": {    "prompt_tokens": 61,    "completion_tokens": 18,    "total_tokens": 79  },  "b3os": {    "route": "local",    "node": "n1"  }}

400 Response

{  "error": {    "message": "prompt_logprobs is not supported on this node",    "type": "invalid_request_error",    "param": "prompt_logprobs",    "code": "unsupported_parameter"  }}

401 Response

{  "error": {    "message": "API key checksum does not match; check that the whole key was copied",    "type": "authentication_error",    "param": null,    "code": "unauthorized"  }}

403 Response

{  "error": {    "message": "<string>",    "type": "<string>",    "param": "<string>",    "code": "<string>"  }}

404 Response

{  "error": {    "message": "gpt-5 is not served on this node; set \"route\":\"gateway\" to use the B3IQ gateway",    "type": "invalid_request_error",    "param": "model",    "code": "model_not_available"  }}

413 Response

{  "error": {    "message": "Request body is 12582912 bytes; this model accepts at most 10485760",    "type": "invalid_request_error",    "param": null,    "code": "payload_too_large"  }}

415 Response

{  "error": {    "message": "expected Content-Type: application/json",    "type": "invalid_request_error",    "param": null,    "code": "unsupported_media_type"  }}

429 Response

{  "error": {    "message": "200000 tokens per minute allowed on b3os/qwen3.5-9b; retry in 9 seconds",    "type": "rate_limit_error",    "param": null,    "code": "rate_limited"  }}

500 Response

{  "error": {    "message": "internal error (request req_01m5t4yg394wegr3j4y01qb363)",    "type": "server_error",    "param": null,    "code": "internal"  }}

503 Response

{  "error": {    "message": "The model runtime on n1 is restarting; retry in 5 seconds",    "type": "server_error",    "param": null,    "code": "infer_backend_unavailable"  }}

POST /v1/infer/chat/completions

Base URL

Bearer Token

Request Body required application/json

{
  "model": "string",
  "messages": [
    {
      "role": "system",
      "content": null,
      "name": "string",
      "tool_calls": [
        {
          "id": "string",
          "type": "function",
          "function": {
            "name": "string",
            "arguments": "string"
          }
        }
      ],
      "tool_call_id": "string",
      "refusal": "string",
      "reasoning_content": "string"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "string",
        "description": "string",
        "parameters": {},
        "strict": true
      }
    }
  ],
  "tool_choice": null,
  "parallel_tool_calls": true,
  "response_format": null,
  "temperature": 0,
  "top_p": 0,
  "max_tokens": 0,
  "max_completion_tokens": 0,
  "stop": null,
  "seed": 0,
  "n": 0,
  "presence_penalty": 0,
  "frequency_penalty": 0,
  "logprobs": true,
  "top_logprobs": 0,
  "user": "string",
  "stream": false,
  "stream_options": {
    "include_usage": true
  },
  "route": "local",
  "top_k": 0,
  "min_p": 0,
  "repetition_penalty": 0,
  "logit_bias": {},
  "chat_template_kwargs": {
    "enable_thinking": true
  },
  "metadata": {},
  "store": true,
  "service_tier": "string"
}

Response

Was this page helpful?

Previous Delete a connection

Next Create a response (OpenAI Responses API)