> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Infer limits and FAQ

Per-model token and request limits, request sizes and stream lengths for inference in the beta, and answers to common questions about the node's models.

## Limits

| Limit | Value |
|---|---|
| Tokens and requests per minute | Per model: `b3os.limits.tpm` and `b3os.limits.rpm` in `GET /v1/infer/models` |
| Largest request | Per model: `b3os.max_request_bytes`; the chat endpoint accepts at most 10 MiB |
| Chat stream length | 10 minutes |
| Choices per request (`n`) | 1 |
| Speech input | 4,096 characters per request; request body 64 KiB |

Going over a model's rate returns `429 rate_limited` with `Retry-After`.

## FAQ

**Which model does the node serve?**

`GET /v1/infer/models` says, with each model's context length, quantization and features. The beta runs one 8B-class instruct model with tool calling and JSON Schema output, and a Kokoro speech model. Write code against the IDs in that list rather than fixed names.
  **Does inference cost anything in the beta?**

No. Tokens are metered so you can see your usage, and each model's tokens-per-minute limit applies.
  **I need a larger model. What are my options?**

Set `"route": "gateway"` to send a request for another model to the B3IQ gateway. B3OS never does this on its own: without the flag, an unknown model fails with `model_not_available`. You can also call any provider you like from your agent; B3OS tools work with any model.
  **Can I use the Anthropic SDK?**

Not yet. An Anthropic Messages endpoint is planned. Today the surface is OpenAI's.
  **Why do I get infer_backend_unavailable?**

The model process is restarting or failing. The response carries `Retry-After`; try again after it.
  **Why does code in my sandbox get forbidden from infer?**

The sandbox's grant lacks the scope. Create it with `api.scopes` including `infer:run` (and `infer:read` to list models).