Messages

POST /v1/infer/v1/messages is the Anthropic Messages API on the node's model. Point an Anthropic SDK at https://beta-api.b3os.org/v1/infer with your B3OS key as the auth token, and set B3OS_MODEL to a chat model from GET /v1/infer/models (setup). Anthropic compatibility lists every field the node supports, ignores or refuses.

Streaming

ts
import Anthropic from "@anthropic-ai/sdk";const client = new Anthropic({ authToken: process.env.B3OS_API_KEY, baseURL: "https://beta-api.b3os.org/v1/infer" });const stream = client.messages.stream({  model: process.env.B3OS_MODEL!,  max_tokens: 256,  messages: [{ role: "user", content: "Count from 1 to 5." }],});stream.on("text", (text) => process.stdout.write(text));const message = await stream.finalMessage();console.log("\n", message.usage);

python
import osimport anthropicclient = anthropic.Anthropic(auth_token=os.environ["B3OS_API_KEY"], base_url="https://beta-api.b3os.org/v1/infer")with client.messages.stream(    model=os.environ["B3OS_MODEL"],    max_tokens=256,    messages=[{"role": "user", "content": "Count from 1 to 5."}],) as stream:    for text in stream.text_stream:        print(text, end="", flush=True)    print("\n", stream.get_final_message().usage)

bash
curl -sN https://beta-api.b3os.org/v1/infer/v1/messages \  -H "Authorization: Bearer $B3OS_API_KEY" \  -H "anthropic-version: 2023-06-01" \  -H "Content-Type: application/json" \  -d "{\"model\": \"$B3OS_MODEL\", \"max_tokens\": 256, \"stream\": true, \"messages\": [{\"role\": \"user\", \"content\": \"Count from 1 to 5.\"}]}"

The events are Anthropic's: message_start, content_block_start, content_block_delta, content_block_stop, message_delta and message_stop. A stream that fails after it started ends with an error event; one that fails before the first token is a plain HTTP error. Server-Timing carries ttft, the time to the first generated token on the node.

Tool use

Define custom tools with input_schema. When the model wants one it stops with stop_reason: "tool_use"; run the tool and send its output back as a tool_result block in the next user turn.

python
tools = [{    "name": "get_weather",    "description": "Current weather for a city.",    "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},}]messages = [{"role": "user", "content": "What is the weather in Eugene, Oregon? Use the tool."}]first = client.messages.create(model=os.environ["B3OS_MODEL"], max_tokens=512, tools=tools, messages=messages)call = next(b for b in first.content if b.type == "tool_use")messages += [    {"role": "assistant", "content": first.content},    {"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "content": "52F and raining"}]},]answer = client.messages.create(model=os.environ["B3OS_MODEL"], max_tokens=512, tools=tools, messages=messages)print(answer.content[-1].text)

ts
const tools: Anthropic.Tool[] = [{  name: "get_weather",  description: "Current weather for a city.",  input_schema: { type: "object", properties: { city: { type: "string" } }, required: ["city"] },}];const messages: Anthropic.MessageParam[] = [  { role: "user", content: "What is the weather in Eugene, Oregon? Use the tool." },];const first = await client.messages.create({ model: process.env.B3OS_MODEL!, max_tokens: 512, tools, messages });const call = first.content.find((b) => b.type === "tool_use")!;messages.push(  { role: "assistant", content: first.content },  { role: "user", content: [{ type: "tool_result", tool_use_id: call.id, content: "52F and raining" }] },);const answer = await client.messages.create({ model: process.env.B3OS_MODEL!, max_tokens: 512, tools, messages });console.log(answer.content.at(-1));

tool_choice takes auto, any, none or {"type": "tool", "name": ...}, with disable_parallel_tool_use. Tools Anthropic runs on its side (web_search, bash, code_execution and the like) are refused: the node runs custom tools only.

Extended thinking

thinking with type: "enabled" turns the model's reasoning on; budget_tokens (at least 1,024, and below max_tokens) caps it. The reasoning comes back in thinking blocks before the answer, and goes back with its turn when you continue the conversation.

python
message = client.messages.create(    model=os.environ["B3OS_MODEL"],    max_tokens=4096,    thinking={"type": "enabled", "budget_tokens": 1024},    messages=[{"role": "user", "content": "Is 391 prime?"}],)for block in message.content:    print(block.type, block.thinking if block.type == "thinking" else block.text)

A budget that does not fit below max_tokens is refused before the model runs:

bash
curl -s -X POST 'https://beta-api.b3os.org/v1/infer/v1/messages' \  -H "Authorization: Bearer $B3OS_API_KEY" \  -H 'content-type: application/json' \  -H 'anthropic-version: 2023-06-01' \  -d '{"model":"b3os/qwen3.5-9b","max_tokens":1024,"thinking":{"type":"enabled","budget_tokens":1024},"messages":[{"role":"user","content":"Is 391 prime?"}]}'

json
{  "b3os": {    "code": "invalid_request",    "param": "thinking.budget_tokens"  },  "error": {    "message": "must be less than max_tokens",    "type": "invalid_request_error"  },  "request_id": "req_01m69wb49g7hqsgt49zg9sngy4",  "type": "error"}

http
server-timing: total;dur=0.121x-request-id: req_01m69wb49g7hqsgt49zg9sngy4

adaptive turns the reasoning on with no budget (only max_tokens bounds it), and disabled, the default, answers without reasoning. output_config.effort is accepted and has no effect.

Long sessions

The node clears nothing from a conversation. context_management (context editing) is accepted and ignored, so every turn you send stays in the prompt, and a long agent session grows until the prompt no longer fits the model's 32,768-token context. That request gets 400 context_length_exceeded. Summarize or drop old turns in your client before then; POST /v1/infer/v1/messages/count_tokens tells you a request's size for free.

Instructions go in the top-level system parameter. A system message inside messages, which some agents send in the middle of a session, is refused with 400 invalid_request rather than moved, because moving it would change what the model reads at that turn. Fold such instructions into the next user turn, or into system when the session starts.

Limits

Messages share chat completions' admission and limits: the estimated prompt plus max_tokens counts against your organization's tokens per minute for the model, and requests per minute and requests in flight apply. Going over answers 429 rate_limited with Retry-After. See Infer limits. Input and output tokens are metered as infer.input_tokens and infer.output_tokens.

json
{  "model": "b3os/qwen3.5-9b",  "max_tokens": 512,  "messages": [{ "role": "user", "content": "What is the weather in Eugene, Oregon? Use the tool." }],  "tools": [    {      "name": "get_weather",      "description": "Current weather for a city.",      "input_schema": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] }    }  ]}