A sandboxed agent answers out loud
Code in a sandbox lets the node's model read a page through vision and speaks its answer from the first sentence, timed from the question to the first audio.
A voice agent with every part on one node. Grant a sandbox infer:run and vision:run, and the code inside it can
call the model, the scraper and the speech model with no key: the sandbox gets OPENAI_BASE_URL, a placeholder
OPENAI_API_KEY and B3OS_MODEL, and B3OS answers at http://127.0.0.1:7700. The agent (agent.mjs in the
recipe) asks the model which tasks on the demo app's launch board are in progress and which of them are engineering
work. The model reads the board with read_page (a vision/scrape of the page as Markdown), then calls answer
with one or two sentences meant to be heard, followed by the task titles. The model's calls stream, and the spoken
part comes first, so the agent does not wait for the whole answer: as soon as the first sentence is complete it goes
to the speech model as a stream, while the model is still writing the rest, and the rest is spoken after it.
jsconst r = await ask((call) => { // call.arguments grows as the model writes: {"spoken": "Four tasks are in progress. Two of them ... const so = spokenSoFar(call.arguments); const sentence = so && firstSentence(so); if (sentence && !firstSpeech) firstSpeech = speak(sentence); // POST /v1/infer/audio/speech, stream_format "audio"});
The agent's own code then reads the board's in-progress column from the same Markdown, so the titles the model gave
are checked against the page and not against the model, and what is said aloud has to name each of those tasks. question_to_first_audio_ms below runs from the first
model call to the first audio bytes, measured inside the sandbox; speech_started_ms is when the first sentence
went to the speech model and answer_done_ms when the model's answer was complete.
Verified 2026-11-02: this recipe's last run passed against https://beta-api.b3os.org, label live-recorded (recorded from production), from commit 6f393285. The recipe harness runs it again, and a failed run takes this page out of the cookbook.
Run it
Each tab creates the sandbox with the grant, copies agent.mjs in, runs it, downloads the answer as answer.mp3
and deletes the sandbox.
bashSBX=$(curl -s https://beta-api.b3os.org/v1/sandboxes \ -H "Authorization: Bearer $B3OS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"template": "base", "timeout_s": 300, "api": {"scopes": ["infer:run", "vision:run"], "spend_cap_usd": 1}}' \ | jq -r .id)curl -s -X PUT "https://beta-api.b3os.org/v1/sandboxes/$SBX/files?path=/tmp/agent.mjs" \ -H "Authorization: Bearer $B3OS_API_KEY" \ -H "Content-Type: application/octet-stream" \ --data-binary @agent.mjscurl -s "https://beta-api.b3os.org/v1/sandboxes/$SBX/exec" \ -H "Authorization: Bearer $B3OS_API_KEY" \ -H "Content-Type: application/json" \ -d '{"cmd": "node /tmp/agent.mjs", "timeout_ms": 60000}' | jq -r .stdoutcurl -s "https://beta-api.b3os.org/v1/sandboxes/$SBX/files?path=/tmp/answer.mp3" \ -H "Authorization: Bearer $B3OS_API_KEY" -o answer.mp3curl -s -X DELETE "https://beta-api.b3os.org/v1/sandboxes/$SBX" -H "Authorization: Bearer $B3OS_API_KEY"
tsimport { readFileSync, writeFileSync } from "node:fs";const API = "https://beta-api.b3os.org";const auth = { authorization: `Bearer ${process.env.B3OS_API_KEY}` };const json = { ...auth, "content-type": "application/json" };const sbx: any = await ( await fetch(`${API}/v1/sandboxes`, { method: "POST", headers: json, body: JSON.stringify({ template: "base", timeout_s: 300, api: { scopes: ["infer:run", "vision:run"], spend_cap_usd: 1 }, }), })).json();try { await fetch(`${API}/v1/sandboxes/${sbx.id}/files?path=/tmp/agent.mjs`, { method: "PUT", headers: { ...auth, "content-type": "application/octet-stream" }, body: readFileSync("agent.mjs"), }); const exec: any = await ( await fetch(`${API}/v1/sandboxes/${sbx.id}/exec`, { method: "POST", headers: json, body: JSON.stringify({ cmd: "node /tmp/agent.mjs", timeout_ms: 60000 }), }) ).json(); console.log(exec.stdout); const mp3 = await fetch(`${API}/v1/sandboxes/${sbx.id}/files?path=/tmp/answer.mp3`, { headers: auth }); writeFileSync("answer.mp3", Buffer.from(await mp3.arrayBuffer()));} finally { await fetch(`${API}/v1/sandboxes/${sbx.id}`, { method: "DELETE", headers: auth });}
pythonimport osimport requestsAPI = "https://beta-api.b3os.org"s = requests.Session()s.headers["Authorization"] = f"Bearer {os.environ['B3OS_API_KEY']}"sbx = s.post(f"{API}/v1/sandboxes", json={ "template": "base", "timeout_s": 300, "api": {"scopes": ["infer:run", "vision:run"], "spend_cap_usd": 1},}).json()try: with open("agent.mjs", "rb") as f: s.put(f"{API}/v1/sandboxes/{sbx['id']}/files", params={"path": "/tmp/agent.mjs"}, data=f.read(), headers={"Content-Type": "application/octet-stream"}) run = s.post(f"{API}/v1/sandboxes/{sbx['id']}/exec", json={"cmd": "node /tmp/agent.mjs", "timeout_ms": 60000}).json() print(run["stdout"]) mp3 = s.get(f"{API}/v1/sandboxes/{sbx['id']}/files", params={"path": "/tmp/answer.mp3"}) with open("answer.mp3", "wb") as f: f.write(mp3.content)finally: s.delete(f"{API}/v1/sandboxes/{sbx['id']}")
On the remote B3OS MCP server, sandbox_create takes the same body as the API, api grant included.
Ask your agent:
textCreate a base sandbox that may call the model and scrape pages (infer:run, vision:run, spend cap 1 USD), writeagent.mjs into /tmp, run it with node, show me its output, then delete the sandbox.
The agent calls sandbox_create, sandbox_write_file, sandbox_exec and sandbox_delete. The audio stays in the
sandbox's /tmp/answer.mp3 until it is deleted; the curl tab's files call downloads it.
model_ms, scrape_ms and the speech figures below are measured inside the sandbox, on the same machine as the
model, the scraper and the speech model. speech_ttfa_server_ms is the node's own ttfa from the speech response's
Server-Timing. The demo app is a fictional workspace with made-up tasks.
Owner: Docs. Products: Docs, Sandboxes, Vision, Infer.
Output of the last green run
bash$ node run.mjs# exit 0
json{ "model": "b3os/qwen3.5-9b", "page_url": "https://beta.b3os.org/demo/", "question": "Which tasks on the launch plan board are in progress, and which of them are engineering work?", "turns": 2, "moves": [ "read_page", "answer" ], "in_progress": [ "Build the new navigation", "Redesign the sign-up form", "Set up redirects from old URLs", "Write the release notes" ], "engineering": [ "Build the new navigation", "Set up redirects from old URLs" ], "spoken": "The tasks in progress are Build the new navigation, Redesign the sign-up form, Set up redirects from old URLs, and Write the release notes. The engineering tasks are Build the new navigation and Set up redirects from old URLs.", "page_in_progress": [ "Build the new navigation", "Redesign the sign-up form", "Set up redirects from old URLs", "Write the release notes" ], "page_engineering": [ "Build the new navigation", "Set up redirects from old URLs" ], "matches_page": true, "spoken_names_all": true, "read_sha256": "db704a06b7263ddb6c1ee45b3015930241faeffbe0379976a8cc62df95d88cfd", "model_ms": [ 203.6, 676.4 ], "scrape_ms": [ 11.9 ], "scrape_tier": "cache", "voice": "af_heart", "speech_parts": 2, "speech_started_ms": 468, "answer_done_ms": 892.8, "speech_first_audio_ms": 116.5, "speech_ttfa_server_ms": 113.487, "speech_ms": 198.2, "question_to_first_audio_ms": 584.8, "agent_ms": 950.6, "audio_type": "audio/mpeg", "audio_bytes": 245466, "audio": "https://beta.b3os.org/gallery/recipes/sandbox-voice-agent/7e4cfe7655a76099.mp3", "create_ms": 103, "exec_ms": 1042.9, "deleted": true}
The outputs this recipe publishes (its recipe.json publish.outputs), with workspace identifiers redacted. The run's stdout, stderr and other outputs are not published.
Captured 2026-11-02 from commit
6f393285
, label
live-recorded
(recorded from production). Case
recipes/sandbox-voice-agent
.