> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Messages

Call the node's model with the Anthropic Messages API from the Anthropic SDKs, including streaming, tool use and extended thinking.

`POST /v1/infer/v1/messages` is the Anthropic Messages API on the node's model. Point an Anthropic SDK at `https://beta-api.b3os.org/v1/infer` with your B3OS key as the auth token, and set `B3OS_MODEL` to a chat model from `GET /v1/infer/models` ([setup](/compat/anthropic)). [Anthropic compatibility](/compat/anthropic) lists every field the node supports, ignores or refuses.

## Streaming

**TypeScript**

```ts
    import Anthropic from "@anthropic-ai/sdk";

    const client = new Anthropic({ authToken: process.env.B3OS_API_KEY, baseURL: "https://beta-api.b3os.org/v1/infer" });
    const stream = client.messages.stream({
      model: process.env.B3OS_MODEL!,
      max_tokens: 256,
      messages: [{ role: "user", content: "Count from 1 to 5." }],
    });
    stream.on("text", (text) => process.stdout.write(text));
    const message = await stream.finalMessage();
    console.log("\n", message.usage);
    ```
  **Python**

```python
    import os
    import anthropic

    client = anthropic.Anthropic(auth_token=os.environ["B3OS_API_KEY"], base_url="https://beta-api.b3os.org/v1/infer")
    with client.messages.stream(
        model=os.environ["B3OS_MODEL"],
        max_tokens=256,
        messages=[{"role": "user", "content": "Count from 1 to 5."}],
    ) as stream:
        for text in stream.text_stream:
            print(text, end="", flush=True)
        print("\n", stream.get_final_message().usage)
    ```
  **curl**

```bash
    curl -sN https://beta-api.b3os.org/v1/infer/v1/messages \
      -H "Authorization: Bearer $B3OS_API_KEY" \
      -H "anthropic-version: 2023-06-01" \
      -H "Content-Type: application/json" \
      -d "{\"model\": \"$B3OS_MODEL\", \"max_tokens\": 256, \"stream\": true, \"messages\": [{\"role\": \"user\", \"content\": \"Count from 1 to 5.\"}]}"
    ```

The events are Anthropic's: `message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta` and `message_stop`. A stream that fails after it started ends with an `error` event; one that fails before the first token is a plain HTTP error. `Server-Timing` carries `ttft`, the time to the first generated token on the node.

## Tool use

Define custom tools with `input_schema`. When the model wants one it stops with `stop_reason: "tool_use"`; run the tool and send its output back as a `tool_result` block in the next user turn.

**Python**

```python
    tools = [{
        "name": "get_weather",
        "description": "Current weather for a city.",
        "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
    }]
    messages = [{"role": "user", "content": "What is the weather in Eugene, Oregon? Use the tool."}]
    first = client.messages.create(model=os.environ["B3OS_MODEL"], max_tokens=512, tools=tools, messages=messages)
    call = next(b for b in first.content if b.type == "tool_use")
    messages += [
        {"role": "assistant", "content": first.content},
        {"role": "user", "content": [{"type": "tool_result", "tool_use_id": call.id, "content": "52F and raining"}]},
    ]
    answer = client.messages.create(model=os.environ["B3OS_MODEL"], max_tokens=512, tools=tools, messages=messages)
    print(answer.content[-1].text)
    ```
  **TypeScript**

```ts
    const tools: Anthropic.Tool[] = [{
      name: "get_weather",
      description: "Current weather for a city.",
      input_schema: { type: "object", properties: { city: { type: "string" } }, required: ["city"] },
    }];
    const messages: Anthropic.MessageParam[] = [
      { role: "user", content: "What is the weather in Eugene, Oregon? Use the tool." },
    ];
    const first = await client.messages.create({ model: process.env.B3OS_MODEL!, max_tokens: 512, tools, messages });
    const call = first.content.find((b) => b.type === "tool_use")!;
    messages.push(
      { role: "assistant", content: first.content },
      { role: "user", content: [{ type: "tool_result", tool_use_id: call.id, content: "52F and raining" }] },
    );
    const answer = await client.messages.create({ model: process.env.B3OS_MODEL!, max_tokens: 512, tools, messages });
    console.log(answer.content.at(-1));
    ```

`tool_choice` takes `auto`, `any`, `none` or `{"type": "tool", "name": ...}`, with `disable_parallel_tool_use`. Tools Anthropic runs on its side (`web_search`, `bash`, `code_execution` and the like) are refused: the node runs custom tools only.

## Extended thinking

`thinking` with `type: "enabled"` turns the model's reasoning on; `budget_tokens` (at least 1,024, and below `max_tokens`) caps it. The reasoning comes back in `thinking` blocks before the answer, and goes back with its turn when you continue the conversation.

```python
message = client.messages.create(
    model=os.environ["B3OS_MODEL"],
    max_tokens=4096,
    thinking={"type": "enabled", "budget_tokens": 1024},
    messages=[{"role": "user", "content": "Is 391 prime?"}],
)
for block in message.content:
    print(block.type, block.thinking if block.type == "thinking" else block.text)
```

A budget that does not fit below `max_tokens` is refused before the model runs:

**Request**

```bash
curl -s -X POST 'https://beta-api.b3os.org/v1/infer/v1/messages' \
  -H "Authorization: Bearer $B3OS_API_KEY" \
  -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  -d '{"model":"b3os/qwen3.5-9b","max_tokens":1024,"thinking":{"type":"enabled","budget_tokens":1024},"messages":[{"role":"user","content":"Is 391 prime?"}]}'
```
**Response 400**

```json
{
  "b3os": {
    "code": "invalid_request",
    "param": "thinking.budget_tokens"
  },
  "error": {
    "message": "must be less than max_tokens",
    "type": "invalid_request_error"
  },
  "request_id": "req_01m69wb49g7hqsgt49zg9sngy4",
  "type": "error"
}
```
**Server-Timing**

```http
server-timing: total;dur=0.121
x-request-id: req_01m69wb49g7hqsgt49zg9sngy4
```

Captured 2026-10-31 from commit `6bd0d80a`, label `dev` (recorded on a dev node before this reached production). Case `infer/messages-thinking-budget`.

`adaptive` turns the reasoning on with no budget (only `max_tokens` bounds it), and `disabled`, the default, answers without reasoning. `output_config.effort` is accepted and has no effect.

## Long sessions

The node clears nothing from a conversation. `context_management` (context editing) is accepted and ignored, so every turn you send stays in the prompt, and a long agent session grows until the prompt no longer fits the model's 32,768-token context. That request gets `400` [`context_length_exceeded`](/errors#context_length_exceeded). Summarize or drop old turns in your client before then; `POST /v1/infer/v1/messages/count_tokens` tells you a request's size for free.

Instructions go in the top-level `system` parameter. A `system` message inside `messages`, which some agents send in the middle of a session, is refused with `400` [`invalid_request`](/errors#invalid_request) rather than moved, because moving it would change what the model reads at that turn. Fold such instructions into the next user turn, or into `system` when the session starts.

## Limits

Messages share chat completions' admission and limits: the estimated prompt plus `max_tokens` counts against your organization's tokens per minute for the model, and requests per minute and requests in flight apply. Going over answers `429` [`rate_limited`](/errors#rate_limited) with `Retry-After`. See [Infer limits](/infer/limits). Input and output tokens are metered as `infer.input_tokens` and `infer.output_tokens`.

```json
{
  "model": "b3os/qwen3.5-9b",
  "max_tokens": 512,
  "messages": [{ "role": "user", "content": "What is the weather in Eugene, Oregon? Use the tool." }],
  "tools": [
    {
      "name": "get_weather",
      "description": "Current weather for a city.",
      "input_schema": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] }
    }
  ]
}
```

[Try it in the playground](https://beta.b3os.org/app/playground?op=inferCreateMessage&body=eyJtb2RlbCI6ImIzb3MvcXdlbjMuNS05YiIsIm1heF90b2tlbnMiOjUxMiwibWVzc2FnZXMiOlt7InJvbGUiOiJ1c2VyIiwiY29udGVudCI6IldoYXQgaXMgdGhlIHdlYXRoZXIgaW4gRXVnZW5lLCBPcmVnb24_IFVzZSB0aGUgdG9vbC4ifV0sInRvb2xzIjpbeyJuYW1lIjoiZ2V0X3dlYXRoZXIiLCJkZXNjcmlwdGlvbiI6IkN1cnJlbnQgd2VhdGhlciBmb3IgYSBjaXR5LiIsImlucHV0X3NjaGVtYSI6eyJ0eXBlIjoib2JqZWN0IiwicHJvcGVydGllcyI6eyJjaXR5Ijp7InR5cGUiOiJzdHJpbmcifX0sInJlcXVpcmVkIjpbImNpdHkiXX19XX0)

Opens the dashboard playground with this request filled in (`POST /v1/infer/v1/messages`). The link carries no key.