> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# A narrated app tour, scripted from what vision saw

Vision tries a page's tabs, buttons and links, the model scripts beats from what they showed, and render records the tour with a voice and captions.

A product video from a URL, scripted by the model from what the app really shows. Vision opens the page in a browser
session and takes two `snapshot`s: the outline (headings and controls, one per line, the shape an agent reads) and
the content of `<main>`. Then, for each tab, button and link in the outline, it goes back to the page, clicks it,
waits until the network is quiet and reads `<main>` again. A click qualifies as a move when it shows at least two
lines the page itself does not, so a tab that only says "Nothing here yet" never makes it into the video, and never
when it lands on a page with a password field: the tour does not record a sign-in form.

The model gets the page's content and each move's view, and writes an opening line and up to three beats: a title,
one sentence about what that move shows, and the move, chosen from a JSON schema `enum` of the qualifying moves.
Beats that stay on the page (tabs, buttons) go before links, which leave it. Before anything is recorded, vision
plays the moves in that order from the page, so a move whose element is gone after an earlier one fails the run
instead of the video. render's `videos` then takes the beats as they are: it records the page, clicks each target by
its role and accessible name while the node's speech model says the line, burns in karaoke captions and returns the
MP4, a poster and WebVTT captions. The outputs below give each step's time in the last green run.

Verified 2026-11-01: this recipe's last run passed against `https://beta-api.b3os.org`, label `live-recorded` (recorded from production), from commit `29fca0d1`. The recipe harness runs it again, and a failed run takes this page out of the cookbook.

## Run it

**curl**

```bash
API=https://beta-api.b3os.org
AUTH="Authorization: Bearer $B3OS_API_KEY"
PAGE=https://beta.b3os.org/demo/
MAIN='{"kind": "snapshot", "mode": "full", "root": "main", "max_tokens": 800}'
post() { curl -s "$API$1" -H "$AUTH" -H "Content-Type: application/json" -d @-; }
act() { jq -n --argjson a "$1" '{actions: $a, continue_on_error: true}' | post "/v1/vision/sessions/$SID/act"; }
lines() { sed -E 's/ \[ref=[^]]*\]//g; s/ \[selected\]//g; s/^ +//' | grep -v '^$'; }
PASSWORD='{"kind": "evaluate", "expression": "document.querySelector(\"input[type=password]\") !== null"}'

# 1. vision: the page's outline and content, then what each tab, button and link shows
SID=$(echo '{}' | post /v1/vision/sessions | jq -r .id)
act "[{\"kind\": \"navigate\", \"url\": \"$PAGE\"}, {\"kind\": \"snapshot\", \"mode\": \"skeleton\"}, $MAIN]" > first.json
jq -r '.results[2].snapshot.text' first.json | lines > home.txt
jq -r '.results[1].snapshot.text' first.json | sed -E 's/ \[ref=[^]]*\]//g' \
| grep -oE -- '- (tab|button|link) "[^"]+"( \[(selected|disabled)\])?' | grep -vE '\[(selected|disabled)\]' \
| sed -E 's/^- (tab|button|link) "(.*)"$/click \1 \2/' | awk '!seen[$0]++' > candidates.txt
: > moves.txt
: > views.txt
while read -r kind role name; do
  act "$(jq -n --arg url "$PAGE" --arg role "$role" --arg name "$name" --argjson pw "$PASSWORD" --argjson main "$MAIN" \
    '[{kind: "navigate", url: $url}, {kind: "click", target: {role: $role, name: $name}},
      {kind: "wait", for: {network_idle: true}}, $pw, $main]')" \
  | jq -r 'select(.ok and .results[3].value == false) | .results[4].snapshot.text // empty' | lines > view.txt
  if [ "$(grep -cvxFf home.txt view.txt)" -ge 2 ]; then   # two lines the page does not show, and no password field
    echo "$kind $role $name" >> moves.txt
    { echo; echo "Move \"$kind $role $name\" then shows:"; head -16 view.txt; } >> views.txt
  fi
done < candidates.txt
curl -s -X DELETE "$API/v1/vision/sessions/$SID" -H "$AUTH" > /dev/null

# 2. infer: the script, its moves held to the qualifying ones
{ echo "Page: $PAGE"; echo "What the page shows:"; head -20 home.txt; cat views.txt; } > read.txt
jq -R . moves.txt | jq -s . > moves.json
N=$(jq 'if length > 3 then 3 else length end' moves.json)
jq -n --rawfile read read.txt --slurpfile moves moves.json --argjson n "$N" '{
  model: "b3os/qwen3.5-9b", temperature: 0, max_tokens: 600, chat_template_kwargs: {enable_thinking: false},
  response_format: {type: "json_schema", json_schema: {name: "tour", strict: true, schema: {
    type: "object", additionalProperties: false, required: ["intro", "beats"], properties: {
      intro: {type: "string", maxLength: 160},
      beats: {type: "array", minItems: $n, maxItems: $n, items: {
        type: "object", additionalProperties: false, required: ["title", "say", "move"], properties: {
          title: {type: "string", maxLength: 30}, say: {type: "string", maxLength: 160},
          move: {type: "string", enum: $moves[0]}}}}}}}},
  messages: [
    {role: "system", content: "You script a short narrated tour of a web app for a product video. Write an opening line (at most 16 words) about what the page shows, then \($n) beats. Each beat has a title of two or three words, one sentence of at most 18 words, and one move; use \($n) different moves, and put moves on this page (tabs, buttons) before links, which open other pages. A beat'"'"'s sentence describes what its move shows, using only the lines given for that move. Plain words, no exclamation marks."},
    {role: "user", content: $read}]}' \
| post /v1/infer/chat/completions \
| jq '.choices[0].message.content | fromjson | .beats |= sort_by(.move | startswith("click link "))' > script.json
jq -r '.intro, (.beats[] | "\(.move): \(.say)")' script.json

# 3. render: the beats as they are, a voice and captions
jq --arg page "$PAGE" '{
  source: {url: $page},
  beats: ([{title: "Overview", say: .intro}] + [.beats[] | {title, say, do: [.move | split(" ") as $m
    | {kind: "click", target: {role: $m[1], name: ($m[2:] | join(" "))}}]}]),
  voice: {model: "kokoro-82m", voice: "af_heart"},
  captions: {burn_in: "karaoke", sidecars: ["vtt"]},
  style: {background: "midnight", frame: "browser-dark", cursor: "arrow", captions: "default", click_effect: "ripple"},
  output: {width: 1280, height: 720, fps: 30}, formats: ["mp4", "poster"]}' script.json \
| post "/v1/render/videos?wait=60" > video.json
jq -r '.status, .result.duration_ms' video.json
jq -r '.result.artifacts[] | select(.kind == "mp4") | .url' video.json | xargs curl -s -o tour.mp4
```

`wait=60` holds the request until the video is done, up to 60 seconds; a longer one answers `running`, and
`GET /v1/render/videos/{id}` reports it until it is `succeeded`. The recipe's `run.mjs` also plays the moves in order
in vision before it records.
**TypeScript**

```ts
const API = "https://beta-api.b3os.org";
const PAGE = "https://beta.b3os.org/demo/";
const MAIN = { kind: "snapshot", mode: "full", root: "main", max_tokens: 800 };
const PASSWORD = { kind: "evaluate", expression: "document.querySelector('input[type=password]') !== null" };
const headers = { authorization: `Bearer ${process.env.B3OS_API_KEY}`, "content-type": "application/json" };
const call = async (method: string, path: string, body?: unknown): Promise<any> =>
  (await fetch(`${API}${path}`, { method, headers, body: body === undefined ? undefined : JSON.stringify(body) })).json();
const lines = (text: string) =>
  text
    .replace(/ \[ref=[^\]]*\]/g, "")
    .split("\n")
    .map((l) => l.replace(/ \[selected\]/g, "").trim())
    .filter(Boolean);
const target = (move: string) => {
  const [, role, ...name] = move.split(" ");
  return { role, name: name.join(" ") };
};

// 1. vision: the page's outline and content, then what each tab and link shows
const session = await call("POST", "/v1/vision/sessions", {});
const act = (actions: unknown[]) =>
  call("POST", `/v1/vision/sessions/${session.id}/act`, { actions, continue_on_error: true });
const first = await act([{ kind: "navigate", url: PAGE }, { kind: "snapshot", mode: "skeleton" }, MAIN]);
const home = lines(first.results[2].snapshot.text);
const candidates = [
  ...new Set(
    [...first.results[1].snapshot.text.matchAll(/- (tab|button|link) "([^"]+)"(.*)/g)]
      .filter((m: RegExpMatchArray) => !(m[1] === "tab" && (m[3] ?? "").includes("[selected]")))
      .filter((m: RegExpMatchArray) => !(m[3] ?? "").includes("[disabled]"))
      .map((m: RegExpMatchArray) => `click ${m[1]} ${m[2]}`),
  ),
] as string[];
const moves: { move: string; view: string[] }[] = [];
for (const move of candidates) {
  const r = await act([
    { kind: "navigate", url: PAGE },
    { kind: "click", target: target(move) },
    { kind: "wait", for: { network_idle: true } },
    PASSWORD,
    MAIN,
  ]);
  if (!r.ok || r.results[3].value !== false) continue; // never a page with a password field
  const view = lines(r.results[4].snapshot.text);
  if (view.filter((l) => !home.includes(l)).length >= 2) moves.push({ move, view: view.slice(0, 16) });
}
await call("DELETE", `/v1/vision/sessions/${session.id}`);
const n = Math.min(3, moves.length);

// 2. infer: the script, its moves held to the qualifying ones
const read = [
  `Page: ${PAGE}`,
  "What the page shows:",
  ...home.slice(0, 20),
  ...moves.flatMap((m) => ["", `Move "${m.move}" then shows:`, ...m.view]),
].join("\n");
const chat = await call("POST", "/v1/infer/chat/completions", {
  model: "b3os/qwen3.5-9b",
  temperature: 0,
  max_tokens: 600,
  chat_template_kwargs: { enable_thinking: false },
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "tour",
      strict: true,
      schema: {
        type: "object",
        additionalProperties: false,
        required: ["intro", "beats"],
        properties: {
          intro: { type: "string", maxLength: 160 },
          beats: {
            type: "array",
            minItems: n,
            maxItems: n,
            items: {
              type: "object",
              additionalProperties: false,
              required: ["title", "say", "move"],
              properties: {
                title: { type: "string", maxLength: 30 },
                say: { type: "string", maxLength: 160 },
                move: { type: "string", enum: moves.map((m) => m.move) },
              },
            },
          },
        },
      },
    },
  },
  messages: [
    {
      role: "system",
      content:
        `You script a short narrated tour of a web app for a product video. Write an opening line (at most 16 words) about what the page shows, then ${n} beats. Each beat has a title of two or three words, one sentence of at most 18 words, and one move; use ${n} different moves, and put moves on this page (tabs, buttons) before links, which open other pages. A beat's sentence describes what its move shows, using only the lines given for that move. Plain words, no exclamation marks.`,
    },
    { role: "user", content: read },
  ],
});
const script: { intro: string; beats: { title: string; say: string; move: string }[] } = JSON.parse(
  chat.choices[0].message.content,
);
script.beats.sort((a, b) => Number(a.move.startsWith("click link ")) - Number(b.move.startsWith("click link ")));

// 3. render: the beats, a voice and captions
const video = await call("POST", "/v1/render/videos?wait=60", {
  source: { url: PAGE },
  beats: [
    { title: "Overview", say: script.intro },
    ...script.beats.map((b) => ({ title: b.title, say: b.say, do: [{ kind: "click", target: target(b.move) }] })),
  ],
  voice: { model: "kokoro-82m", voice: "af_heart" },
  captions: { burn_in: "karaoke", sidecars: ["vtt"] },
  style: { background: "midnight", frame: "browser-dark", cursor: "arrow", captions: "default", click_effect: "ripple" },
  output: { width: 1280, height: 720, fps: 30 },
  formats: ["mp4", "poster"],
});
console.log(video.status, video.result?.artifacts.find((a: any) => a.kind === "mp4")?.url);
```
**Python**

```python
import json
import os
import re
import requests

API = "https://beta-api.b3os.org"
PAGE = "https://beta.b3os.org/demo/"
MAIN = {"kind": "snapshot", "mode": "full", "root": "main", "max_tokens": 800}
PASSWORD = {"kind": "evaluate", "expression": "document.querySelector('input[type=password]') !== null"}
s = requests.Session()
s.headers["Authorization"] = f"Bearer {os.environ['B3OS_API_KEY']}"


def lines(text):
    text = re.sub(r" \[ref=[^\]]*\]", "", text)
    return [l.replace(" [selected]", "").strip() for l in text.split("\n") if l.strip()]


def target(move):
    _, role, *name = move.split(" ")
    return {"role": role, "name": " ".join(name)}


# 1. vision: the page's outline and content, then what each tab, button and link shows
sid = s.post(f"{API}/v1/vision/sessions", json={}).json()["id"]
act = lambda actions: s.post(f"{API}/v1/vision/sessions/{sid}/act",
                             json={"actions": actions, "continue_on_error": True}).json()
first = act([{"kind": "navigate", "url": PAGE}, {"kind": "snapshot", "mode": "skeleton"}, MAIN])
home = lines(first["results"][2]["snapshot"]["text"])
candidates = []
for role, name, rest in re.findall(r'- (tab|button|link) "([^"]+)"(.*)', first["results"][1]["snapshot"]["text"]):
    move = f"click {role} {name}"
    skip = (role == "tab" and "[selected]" in rest) or "[disabled]" in rest
    if not skip and move not in candidates:
        candidates.append(move)
moves = []
for move in candidates:
    r = act([{"kind": "navigate", "url": PAGE}, {"kind": "click", "target": target(move)},
             {"kind": "wait", "for": {"network_idle": True}}, PASSWORD, MAIN])
    if r.get("ok") and r["results"][3].get("value") is False:  # never a page with a password field
        view = lines(r["results"][4]["snapshot"]["text"])
        if len([l for l in view if l not in home]) >= 2:
            moves.append({"move": move, "view": view[:16]})
s.delete(f"{API}/v1/vision/sessions/{sid}")
n = min(3, len(moves))

# 2. infer: the script, its moves held to the qualifying ones
read = "\n".join([f"Page: {PAGE}", "What the page shows:", *home[:20],
                  *[x for m in moves for x in ["", f'Move "{m["move"]}" then shows:', *m["view"]]]])
beat = {"type": "object", "additionalProperties": False, "required": ["title", "say", "move"],
        "properties": {"title": {"type": "string", "maxLength": 30}, "say": {"type": "string", "maxLength": 160},
                       "move": {"type": "string", "enum": [m["move"] for m in moves]}}}
schema = {"type": "object", "additionalProperties": False, "required": ["intro", "beats"],
          "properties": {"intro": {"type": "string", "maxLength": 160},
                         "beats": {"type": "array", "minItems": n, "maxItems": n, "items": beat}}}
chat = s.post(f"{API}/v1/infer/chat/completions", json={
    "model": "b3os/qwen3.5-9b", "temperature": 0, "max_tokens": 600,
    "chat_template_kwargs": {"enable_thinking": False},
    "response_format": {"type": "json_schema", "json_schema": {"name": "tour", "strict": True, "schema": schema}},
    "messages": [
        {"role": "system", "content": "You script a short narrated tour of a web app for a product video. Write an "
         f"opening line (at most 16 words) about what the page shows, then {n} beats. Each beat has a title of two "
         f"or three words, one sentence of at most 18 words, and one move; use {n} different moves, and put moves "
         "on this page (tabs, buttons) before links, which open other pages. A beat's sentence describes what its "
         "move shows, using only the lines given for that move. Plain words, no exclamation marks."},
        {"role": "user", "content": read},
    ],
}).json()
script = json.loads(chat["choices"][0]["message"]["content"])
script["beats"].sort(key=lambda b: b["move"].startswith("click link "))

# 3. render: the beats, a voice and captions
video = s.post(f"{API}/v1/render/videos", params={"wait": 60}, json={
    "source": {"url": PAGE},
    "beats": [{"title": "Overview", "say": script["intro"]}]
    + [{"title": b["title"], "say": b["say"], "do": [{"kind": "click", "target": target(b["move"])}]}
       for b in script["beats"]],
    "voice": {"model": "kokoro-82m", "voice": "af_heart"},
    "captions": {"burn_in": "karaoke", "sidecars": ["vtt"]},
    "style": {"background": "midnight", "frame": "browser-dark", "cursor": "arrow", "captions": "default",
              "click_effect": "ripple"},
    "output": {"width": 1280, "height": 720, "fps": 30},
    "formats": ["mp4", "poster"],
}).json()
print(video["status"], next(a["url"] for a in video["result"]["artifacts"] if a["kind"] == "mp4"))
```
**MCP**

On the remote [B3OS MCP server](/sdks/mcp), ask your agent:

```text
Open https://beta.b3os.org/demo/ in a vision session and take a skeleton snapshot and a full snapshot of <main>. For
each tab that is not selected, each button and each link in the outline, go back to the page, click it and take a
full snapshot of <main>; keep the clicks that show at least two things the page itself does not, and none that opens
a sign-in form. Close the session. Script a narrated tour: an opening line about the page, then up to three beats,
each a short title, one sentence about what its click shows (only from that snapshot) and the click, with clicks
that stay on the page before links. Record it with render_video: the page as the source, each
beat's click as a role and name target, the kokoro-82m voice af_heart, karaoke captions, the midnight background and
the browser-dark frame. Give me the MP4 URL.
```

The agent calls `vision_create_session`, `vision_act` once per click and `vision_close_session`, writes the script
itself and calls `render_video`. Nothing holds its script to the clicks it tried but its own reading; the recipe's
schema `enum` is what keeps a scripted run from naming an element the page lacks.

**The narrated tour from the last green run**

[Open the video](https://beta.b3os.org/gallery/recipes/narrated-page-tour/890b4ed7386b2177.mp4) · [Captions (WebVTT)](https://beta.b3os.org/gallery/recipes/narrated-page-tour/808193ec1dc70348.vtt)

Transcript

`0:00` This dashboard shows a launch plan with tabs for board, list, and timeline views.

`0:06` Clicking this button opens a form to add a new task to the list.

`0:11` The billing page lets you choose between monthly or annual payment plans.

`0:17` This page lists the starter, team, and business options with clear links.

Captured 2026-11-01 from commit `29fca0d1`, label `live-recorded` (recorded from production). Case `recipes/narrated-page-tour`.

The video above is the last green run's, recorded in production. The page is B3OS's demo app, a fictional workspace
with made-up data. The recipe pins what the model read: a run whose page or views differ from the published run's is
held for a person to watch before it replaces this video.

Owner: Docs. Products: Docs, Vision, Infer, Render.

## Output of the last green run

**Command**

```bash
$ node run.mjs
# exit 0
```
**Outputs**

```json
{
  "page_url": "https://beta.b3os.org/demo/",
  "page_title": "Launch plan",
  "candidates": 11,
  "moves_allowed": 4,
  "read_sha256": "a27fb40f1a988cf0ea867e64653e8a4b864970aa95585d8874d67736ed60c96e",
  "model": "b3os/qwen3.5-9b",
  "intro": "This dashboard shows a launch plan with tabs for board, list, and timeline views.",
  "beats": [
    {
      "title": "Add New Task",
      "say": "Clicking this button opens a form to add a new task to the list.",
      "move": "click button New task"
    },
    {
      "title": "Choose Your Plan",
      "say": "The billing page lets you choose between monthly or annual payment plans.",
      "move": "click link Billing"
    },
    {
      "title": "View All Tiers",
      "say": "This page lists the starter, team, and business options with clear links.",
      "move": "click link Pricing"
    }
  ],
  "render_status": "succeeded",
  "narration_spoken": true,
  "duration_ms": 23025,
  "video": "https://beta.b3os.org/gallery/recipes/narrated-page-tour/890b4ed7386b2177.mp4",
  "poster": "https://beta.b3os.org/gallery/recipes/narrated-page-tour/50285d7758f85986.jpg",
  "captions": "https://beta.b3os.org/gallery/recipes/narrated-page-tour/808193ec1dc70348.vtt",
  "video_bytes": 1057543,
  "explore_ms": 10031.8,
  "script_ms": 881.9,
  "check_ms": 1867.048,
  "render_ms": 4518.2,
  "total_ms": 18364.4,
  "session_closed": true
}
```

[Artifact: video/mp4](https://beta.b3os.org/gallery/recipes/narrated-page-tour/890b4ed7386b2177.mp4)
![Image 1 of 1 from the last green run of the recipe A narrated app tour, scripted from what vision saw](https://beta.b3os.org/gallery/recipes/narrated-page-tour/50285d7758f85986.jpg)
[Artifact: text/vtt](https://beta.b3os.org/gallery/recipes/narrated-page-tour/808193ec1dc70348.vtt)

The outputs this recipe publishes (its recipe.json publish.outputs), with workspace identifiers redacted. The run's stdout, stderr and other outputs are not published.

Captured 2026-11-01 from commit `29fca0d1`, label `live-recorded` (recorded from production). Case `recipes/narrated-page-tour`.