> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# A sandboxed agent answers out loud

Code in a sandbox lets the node's model read a page through vision and speaks its answer from the first sentence, timed from the question to the first audio.

A voice agent with every part on one node. Grant a sandbox `infer:run` and `vision:run`, and the code inside it can
call the model, the scraper and the speech model with no key: the sandbox gets `OPENAI_BASE_URL`, a placeholder
`OPENAI_API_KEY` and `B3OS_MODEL`, and B3OS answers at `http://127.0.0.1:7700`. The agent (`agent.mjs` in the
recipe) asks the model which tasks on the demo app's launch board are in progress and which of them are engineering
work. The model reads the board with `read_page` (a `vision/scrape` of the page as Markdown), then calls `answer`
with one or two sentences meant to be heard, followed by the task titles. The model's calls stream, and the spoken
part comes first, so the agent does not wait for the whole answer: as soon as the first sentence is complete it goes
to the speech model as a stream, while the model is still writing the rest, and the rest is spoken after it.

```js
const r = await ask((call) => {
  // call.arguments grows as the model writes: {"spoken": "Four tasks are in progress. Two of them ...
  const so = spokenSoFar(call.arguments);
  const sentence = so && firstSentence(so);
  if (sentence && !firstSpeech) firstSpeech = speak(sentence); // POST /v1/infer/audio/speech, stream_format "audio"
});
```

The agent's own code then reads the board's in-progress column from the same Markdown, so the titles the model gave
are checked against the page and not against the model, and what is said aloud has to name each of those tasks. `question_to_first_audio_ms` below runs from the first
model call to the first audio bytes, measured inside the sandbox; `speech_started_ms` is when the first sentence
went to the speech model and `answer_done_ms` when the model's answer was complete.

Verified 2026-11-02: this recipe's last run passed against `https://beta-api.b3os.org`, label `live-recorded` (recorded from production), from commit `6f393285`. The recipe harness runs it again, and a failed run takes this page out of the cookbook.

## Run it

Each tab creates the sandbox with the grant, copies `agent.mjs` in, runs it, downloads the answer as `answer.mp3`
and deletes the sandbox.

**curl**

```bash
SBX=$(curl -s https://beta-api.b3os.org/v1/sandboxes \
  -H "Authorization: Bearer $B3OS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"template": "base", "timeout_s": 300, "api": {"scopes": ["infer:run", "vision:run"], "spend_cap_usd": 1}}' \
  | jq -r .id)

curl -s -X PUT "https://beta-api.b3os.org/v1/sandboxes/$SBX/files?path=/tmp/agent.mjs" \
  -H "Authorization: Bearer $B3OS_API_KEY" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @agent.mjs

curl -s "https://beta-api.b3os.org/v1/sandboxes/$SBX/exec" \
  -H "Authorization: Bearer $B3OS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"cmd": "node /tmp/agent.mjs", "timeout_ms": 60000}' | jq -r .stdout

curl -s "https://beta-api.b3os.org/v1/sandboxes/$SBX/files?path=/tmp/answer.mp3" \
  -H "Authorization: Bearer $B3OS_API_KEY" -o answer.mp3

curl -s -X DELETE "https://beta-api.b3os.org/v1/sandboxes/$SBX" -H "Authorization: Bearer $B3OS_API_KEY"
```
**TypeScript**

```ts
import { readFileSync, writeFileSync } from "node:fs";

const API = "https://beta-api.b3os.org";
const auth = { authorization: `Bearer ${process.env.B3OS_API_KEY}` };
const json = { ...auth, "content-type": "application/json" };

const sbx: any = await (
  await fetch(`${API}/v1/sandboxes`, {
    method: "POST",
    headers: json,
    body: JSON.stringify({
      template: "base",
      timeout_s: 300,
      api: { scopes: ["infer:run", "vision:run"], spend_cap_usd: 1 },
    }),
  })
).json();
try {
  await fetch(`${API}/v1/sandboxes/${sbx.id}/files?path=/tmp/agent.mjs`, {
    method: "PUT",
    headers: { ...auth, "content-type": "application/octet-stream" },
    body: readFileSync("agent.mjs"),
  });
  const exec: any = await (
    await fetch(`${API}/v1/sandboxes/${sbx.id}/exec`, {
      method: "POST",
      headers: json,
      body: JSON.stringify({ cmd: "node /tmp/agent.mjs", timeout_ms: 60000 }),
    })
  ).json();
  console.log(exec.stdout);
  const mp3 = await fetch(`${API}/v1/sandboxes/${sbx.id}/files?path=/tmp/answer.mp3`, { headers: auth });
  writeFileSync("answer.mp3", Buffer.from(await mp3.arrayBuffer()));
} finally {
  await fetch(`${API}/v1/sandboxes/${sbx.id}`, { method: "DELETE", headers: auth });
}
```
**Python**

```python
import os
import requests

API = "https://beta-api.b3os.org"
s = requests.Session()
s.headers["Authorization"] = f"Bearer {os.environ['B3OS_API_KEY']}"

sbx = s.post(f"{API}/v1/sandboxes", json={
    "template": "base",
    "timeout_s": 300,
    "api": {"scopes": ["infer:run", "vision:run"], "spend_cap_usd": 1},
}).json()
try:
    with open("agent.mjs", "rb") as f:
        s.put(f"{API}/v1/sandboxes/{sbx['id']}/files", params={"path": "/tmp/agent.mjs"}, data=f.read(),
              headers={"Content-Type": "application/octet-stream"})
    run = s.post(f"{API}/v1/sandboxes/{sbx['id']}/exec",
                 json={"cmd": "node /tmp/agent.mjs", "timeout_ms": 60000}).json()
    print(run["stdout"])
    mp3 = s.get(f"{API}/v1/sandboxes/{sbx['id']}/files", params={"path": "/tmp/answer.mp3"})
    with open("answer.mp3", "wb") as f:
        f.write(mp3.content)
finally:
    s.delete(f"{API}/v1/sandboxes/{sbx['id']}")
```
**MCP**

On the remote [B3OS MCP server](/sdks/mcp), `sandbox_create` takes the same body as the API, `api` grant included.
Ask your agent:

```text
Create a base sandbox that may call the model and scrape pages (infer:run, vision:run, spend cap 1 USD), write
agent.mjs into /tmp, run it with node, show me its output, then delete the sandbox.
```

The agent calls `sandbox_create`, `sandbox_write_file`, `sandbox_exec` and `sandbox_delete`. The audio stays in the
sandbox's `/tmp/answer.mp3` until it is deleted; the curl tab's `files` call downloads it.

`model_ms`, `scrape_ms` and the speech figures below are measured inside the sandbox, on the same machine as the
model, the scraper and the speech model. `speech_ttfa_server_ms` is the node's own `ttfa` from the speech response's
`Server-Timing`. The demo app is a fictional workspace with made-up tasks.

Owner: Docs. Products: Docs, Sandboxes, Vision, Infer.

## Output of the last green run

**Command**

```bash
$ node run.mjs
# exit 0
```
**Outputs**

```json
{
  "model": "b3os/qwen3.5-9b",
  "page_url": "https://beta.b3os.org/demo/",
  "question": "Which tasks on the launch plan board are in progress, and which of them are engineering work?",
  "turns": 2,
  "moves": [
    "read_page",
    "answer"
  ],
  "in_progress": [
    "Build the new navigation",
    "Redesign the sign-up form",
    "Set up redirects from old URLs",
    "Write the release notes"
  ],
  "engineering": [
    "Build the new navigation",
    "Set up redirects from old URLs"
  ],
  "spoken": "The tasks in progress are Build the new navigation, Redesign the sign-up form, Set up redirects from old URLs, and Write the release notes. The engineering tasks are Build the new navigation and Set up redirects from old URLs.",
  "page_in_progress": [
    "Build the new navigation",
    "Redesign the sign-up form",
    "Set up redirects from old URLs",
    "Write the release notes"
  ],
  "page_engineering": [
    "Build the new navigation",
    "Set up redirects from old URLs"
  ],
  "matches_page": true,
  "spoken_names_all": true,
  "read_sha256": "db704a06b7263ddb6c1ee45b3015930241faeffbe0379976a8cc62df95d88cfd",
  "model_ms": [
    203.6,
    676.4
  ],
  "scrape_ms": [
    11.9
  ],
  "scrape_tier": "cache",
  "voice": "af_heart",
  "speech_parts": 2,
  "speech_started_ms": 468,
  "answer_done_ms": 892.8,
  "speech_first_audio_ms": 116.5,
  "speech_ttfa_server_ms": 113.487,
  "speech_ms": 198.2,
  "question_to_first_audio_ms": 584.8,
  "agent_ms": 950.6,
  "audio_type": "audio/mpeg",
  "audio_bytes": 245466,
  "audio": "https://beta.b3os.org/gallery/recipes/sandbox-voice-agent/7e4cfe7655a76099.mp3",
  "create_ms": 103,
  "exec_ms": 1042.9,
  "deleted": true
}
```



The outputs this recipe publishes (its recipe.json publish.outputs), with workspace identifiers redacted. The run's stdout, stderr and other outputs are not published.

Captured 2026-11-02 from commit `6f393285`, label `live-recorded` (recorded from production). Case `recipes/sandbox-voice-agent`.