Create a message (Anthropic Messages API)

The Anthropic Messages API on the node's own model, so Anthropic SDKs and agents built on them run against it with only the base URL and the credential changed: set `ANTHROPIC_BASE_URL` (or `base_url`) to `https://beta-api.b3os.org/v1/infer`, the same base URL as the OpenAI SDKs (`http://127.0.0.1:7700/v1/infer` inside a sandbox; the SDKs add `/v1/messages`), and send the B3OS key as a bearer token (`ANTHROPIC_AUTH_TOKEN`, or `auth_token` in the SDKs). Text, images (`base64`, or `url` sources fetched by the node through its egress guard), custom tools with `tool_use` and `tool_result` blocks, `tool_choice`, `thinking` (the reasoning arrives in `thinking` blocks; `budget_tokens` caps it), JSON Schema output through `output_config.format`, stop sequences, a trailing assistant turn that the model continues, and streaming events (`message_start` ... `message_stop`, exactly as Anthropic sends them). Models are those of `GET /v1/infer/models`; there is no gateway route here. Parameters this node does not take return `400 unsupported_parameter` (tools Anthropic defines or runs, such as `web_search` or `bash`, `container`, `mcp_servers`, document and file sources). `cache_control`, `citations`, `metadata`, `service_tier`, `output_config.effort` and `thinking.display` are accepted and ignored (the runtime caches prompt prefixes on its own). Errors use the Anthropic shape (`{"type": "error", "error": {...}}`, with the B3OS code under `b3os`), except those raised before the request reaches the infer service (authentication, scopes, body size), which use the OpenAI shape of the rest of `/v1/infer`; Anthropic SDKs map both by HTTP status. A stream that fails after it started ends with an `error` event; one that fails before the first token is a plain HTTP error. The response head of a stream waits for the first generated token, so `Server-Timing` can carry `ttft`. Bills input and output tokens, with the admission and limits of chat completions (the estimated prompt plus `max_tokens`, settled with the real counts; an abandoned stream is billed to its last token).

POST /v1/infer/v1/messages

Authentication

Bearer Token (b3os_sk_ API key or b3os_pat_ token) API Key (cookie: __Host-b3os_session)

Request Body required

application/json

model string REQUIRED

messages (object | object)[] REQUIRED

Array of:

One of:

Option 1

role string REQUIRED

Enum: user

content string | (object | object | object)[] REQUIRED

One of:

Option 1string

Option 2

Array of:

One of:

Option 1...

Option 2...

Option 3...

Option 2

role string REQUIRED

Enum: assistant

content string | (object | object | object | object)[] REQUIRED

One of:

Option 1string

Option 2

Array of:

One of:

Option 1...

Option 2...

Option 3...

Option 4...

max_tokens integer REQUIRED

system string | null | object[]

The system prompt; text blocks are joined with blank lines.

One of:

Option 1string | null

Option 2

Array of:

type string REQUIRED

Enum: text

text string REQUIRED

cache_control object | null

Accepted and ignored.

type string

ttl string

citations object[] | null

Accepted and ignored.

Array of: object

tools object[] | null

Array of:

type string | null

Enum: custom, null

name string REQUIRED

Unique within `tools`.

description string | null

input_schema object REQUIRED

A JSON Schema object (at most 64 KiB serialized and 32 levels deep).

strict boolean | null

cache_control object | null

Accepted and ignored.

type string

ttl string

tool_choice object | object | object | null

One of:

Option 1

type string REQUIRED

`any` makes the model call a tool (it needs `tools`).

Enum: auto, any

disable_parallel_tool_use boolean | null

Option 2

type string REQUIRED

Enum: tool

name string REQUIRED

One of `tools`.

disable_parallel_tool_use boolean | null

Option 3

type string REQUIRED

Enum: none

thinking object | object | object | null

`enabled` (with `budget_tokens`, at least 1024 and less than `max_tokens`) and `adaptive` let the model reason first, in `thinking` blocks; `disabled`, the default, does not. `display` is accepted and ignored.

One of:

Option 1

type string REQUIRED

Enum: enabled

budget_tokens integer REQUIRED

display string | null

Enum: summarized, omitted, updates, null

Option 2

type string REQUIRED

Enum: adaptive

display string | null

Enum: summarized, omitted, updates, null

Option 3

type string REQUIRED

Enum: disabled

output_config object | null

format object | null

The answer is JSON matching `schema` (at most 64 KiB serialized and 32 levels deep).

type string REQUIRED

Enum: json_schema

schema object REQUIRED

A JSON Schema object, as Anthropic requires.

effort string | null

Accepted and ignored.

Enum: low, medium, high, xhigh, max, null

stop_sequences string[] | null

Array of: string

stream boolean | null

temperature number | null

top_p number | null

Nucleus sampling. `0` keeps only the most probable token (passed to the runtime as 1e-9, the same thing, since vLLM takes values above 0 only).

top_k integer | null

metadata object | null

Accepted and ignored.

user_id string | null

service_tier string | null

Accepted and ignored.

Responses

200 The message, or the event stream with `stream: true`.

application/json

id string REQUIRED

type string REQUIRED

Enum: message

role string REQUIRED

Enum: assistant

content object[] REQUIRED

Array of:

type string REQUIRED

Enum: text, tool_use, thinking

text string

id string

name string

input object

thinking string

signature string

model string REQUIRED

stop_reason string REQUIRED

Enum: end_turn, max_tokens, stop_sequence, tool_use, refusal

stop_sequence string | null REQUIRED

The stop sequence that ended the answer.

usage object REQUIRED

input_tokens integer REQUIRED

output_tokens integer REQUIRED

b3os object

B3OS extension; OpenAI SDKs ignore it. On a stream it rides on the last chunk: the usage chunk with `stream_options.include_usage`, else the chunk with `finish_reason`.

route string

Enum: local, gateway

node string

timing object

Server-side milliseconds from the request head (`ttft_ms` on streams, `model_ms` without).

ttft_ms number

model_ms number

total_ms number

400 The request is invalid (`invalid_request`, `unsupported_parameter`, `context_length_exceeded`, `unsupported_media_type`).

401 Missing or invalid credentials (`unauthorized`), checked before the request reaches the infer service (OpenAI shape).

403 The key lacks `infer:run` (`forbidden`, OpenAI shape), or the egress guard refused an image URL (`egress_denied`).

404 The model is not served on this node (`model_not_available`).

413 The request body exceeds the model's `max_request_bytes` (`payload_too_large`).

415 The body is not JSON (`unsupported_media_type`): send `Content-Type: application/json`.

429 A limit applies (`rate_limited` for requests, tokens or remote image bytes per minute, `concurrency_limited` for requests in flight for the organization).

500 Something failed inside B3OS (`internal`).

503 The model runtime is restarting, overloaded or not set up on this node (`infer_backend_unavailable`), or the node is unavailable (`node_unavailable`, OpenAI shape).

curl -X POST 'https://beta-api.b3os.org/v1/infer/v1/messages' \  -H 'Authorization: Bearer YOUR_API_TOKEN' \  -H 'Content-Type: application/json' \  -d '{  "model": "string",  "messages": [    {      "role": "user",      "content": "string"    }  ],  "max_tokens": 1,  "system": "string",  "tools": [    {      "type": "custom",      "name": "string",      "description": "string",      "input_schema": {},      "strict": true,      "cache_control": {        "type": "string",        "ttl": "string"      }    }  ],  "tool_choice": {},  "thinking": {},  "output_config": {    "format": {      "type": "json_schema",      "schema": {}    },    "effort": "low"  },  "stop_sequences": [    "string"  ],  "stream": true,  "temperature": 0,  "top_p": 0,  "top_k": 1,  "metadata": {    "user_id": "string"  },  "service_tier": "string"}'

const response = await fetch('https://beta-api.b3os.org/v1/infer/v1/messages', {  method: 'POST',  headers: {      "Authorization": "Bearer YOUR_API_TOKEN",      "Content-Type": "application/json"  },  body: JSON.stringify({    "model": "string",    "messages": [      {        "role": "user",        "content": "string"      }    ],    "max_tokens": 1,    "system": "string",    "tools": [      {        "type": "custom",        "name": "string",        "description": "string",        "input_schema": {},        "strict": true,        "cache_control": {          "type": "string",          "ttl": "string"        }      }    ],    "tool_choice": {},    "thinking": {},    "output_config": {      "format": {        "type": "json_schema",        "schema": {}      },      "effort": "low"    },    "stop_sequences": [      "string"    ],    "stream": true,    "temperature": 0,    "top_p": 0,    "top_k": 1,    "metadata": {      "user_id": "string"    },    "service_tier": "string"  })});const data = await response.json();console.log(data);

import requestsheaders = {    'Authorization': 'Bearer YOUR_API_TOKEN'}response = requests.post('https://beta-api.b3os.org/v1/infer/v1/messages', headers=headers, json={  "model": "string",  "messages": [    {      "role": "user",      "content": "string"    }  ],  "max_tokens": 1,  "system": "string",  "tools": [    {      "type": "custom",      "name": "string",      "description": "string",      "input_schema": {},      "strict": true,      "cache_control": {        "type": "string",        "ttl": "string"      }    }  ],  "tool_choice": {},  "thinking": {},  "output_config": {    "format": {      "type": "json_schema",      "schema": {}    },    "effort": "low"  },  "stop_sequences": [    "string"  ],  "stream": true,  "temperature": 0,  "top_p": 0,  "top_k": 1,  "metadata": {    "user_id": "string"  },  "service_tier": "string"})print(response.json())

200 Response

{  "id": "msg_9f1c2a7e4b5d4e0f8a6b3c2d1e0f9a8b",  "type": "message",  "role": "assistant",  "model": "b3os/qwen3.5-9b",  "content": [    {      "type": "tool_use",      "id": "chatcmpl-tool-7f3a",      "name": "get_weather",      "input": {        "city": "Eugene"      }    }  ],  "stop_reason": "tool_use",  "stop_sequence": null,  "usage": {    "input_tokens": 61,    "output_tokens": 18  },  "b3os": {    "route": "local",    "node": "n1"  }}

400 Response

{  "type": "error",  "error": {    "type": "invalid_request_error",    "message": "web_search_20250305 is a tool Anthropic defines or runs; this node takes custom tools (name, description, input_schema)"  },  "request_id": "req_01m5t4yg394wegr3j4y01qb363",  "b3os": {    "code": "unsupported_parameter",    "param": "tools.0.type"  }}

401 Response

{  "error": {    "message": "API key checksum does not match; check that the whole key was copied",    "type": "authentication_error",    "param": null,    "code": "unauthorized"  }}

403 Response

{  "type": "error",  "error": {    "type": "invalid_request_error",    "message": "<string>"  },  "request_id": "<string>",  "b3os": {    "code": "<string>",    "param": "<string>"  }}

404 Response

{  "type": "error",  "error": {    "type": "not_found_error",    "message": "claude-sonnet-4-5 is not served on this node; GET /v1/infer/models lists the models that are"  },  "b3os": {    "code": "model_not_available",    "param": "model"  }}

413 Response

{  "type": "error",  "error": {    "type": "request_too_large",    "message": "Request body is 12582912 bytes; b3os/qwen3.5-9b accepts at most 10485760"  },  "b3os": {    "code": "payload_too_large"  }}

415 Response

{  "type": "error",  "error": {    "type": "invalid_request_error",    "message": "expected Content-Type: application/json"  },  "b3os": {    "code": "unsupported_media_type"  }}

429 Response

{  "type": "error",  "error": {    "type": "rate_limit_error",    "message": "200000 tokens per minute allowed on b3os/qwen3.5-9b; retry in 9 seconds"  },  "b3os": {    "code": "rate_limited"  }}

500 Response

{  "type": "error",  "error": {    "type": "api_error",    "message": "internal error (request req_01m5t4yg394wegr3j4y01qb363)"  },  "b3os": {    "code": "internal"  }}

503 Response

{  "type": "error",  "error": {    "type": "overloaded_error",    "message": "The model runtime on n1 is restarting; retry in 5 seconds"  },  "b3os": {    "code": "infer_backend_unavailable"  }}

POST /v1/infer/v1/messages

Base URL

Bearer Token

Request Body required application/json

Response

Was this page helpful?

Previous List voices

Next Count the tokens of a message request