Run browser actions

Runs a batch of browser actions in order on one page and returns every result in one round trip: `navigate`, `click`, `hover`, `type`, `key`, `scroll`, `select` and `wait` (the same kinds and fields as render scripts), plus `evaluate`, `screenshot` and `markdown`. Execution stops at the first failed action unless `continue_on_error` is set; the failed action's result carries the error (for example `target_not_found`) and later actions are reported as `skipped`. The batch must finish within `timeout_ms`.

POST /v1/vision/sessions/{session_id}/act
Runs a batch of browser actions in order on one page and returns every result in one round trip: `navigate`, `click`, `hover`, `type`, `key`, `scroll`, `select` and `wait` (the same kinds and fields as render scripts), plus `evaluate`, `screenshot` and `markdown`. Execution stops at the first failed action unless `continue_on_error` is set; the failed action's result carries the error (for example `target_not_found`) and later actions are reported as `skipped`. The batch must finish within `timeout_ms`.

Authentication

Bearer Token (b3os_sk_ API key or b3os_pat_ token) API Key (cookie: __Host-b3os_session)

Path Parameters

session_id string required path
The session ID (`brs_…`).
Example: "brs_01m5t3y30wcksxf22t9gwk9fwb"

Request Body required

application/json
actions (object | object | object | object | object | object | object | object | object | object | object)[] REQUIRED
Array of:
One of:
Option 1
kind string REQUIRED
Enum: navigate
url string (uri) REQUIRED
wait_until string
Enum: load, domcontentloaded, networkidle
timeout_ms integer
Option 2
kind string REQUIRED
Enum: click
target unknown | unknown | unknown REQUIRED
How an action finds its element (the same rules as render timelines, `timeline.schema.json#/$defs/target`): candidates match `selector` (CSS), `text` (visible text, case-insensitive substring unless `exact`) or `role` plus accessible `name`, limited to `within`; the best-scored candidate wins, or the `nth` match when set. An element that stays outside the viewport after scrolling it into view fails with `target_not_found`.
Any of:
Option 1unknown
Option 2unknown
Option 3unknown
button string
Enum: left, right, middle
count integer
modifiers string[]
Modifier keys held during the action.
Array of: string
timeout_ms integer
How long to wait for the target to appear.
Option 3
kind string REQUIRED
Enum: hover
target unknown | unknown | unknown REQUIRED
How an action finds its element (the same rules as render timelines, `timeline.schema.json#/$defs/target`): candidates match `selector` (CSS), `text` (visible text, case-insensitive substring unless `exact`) or `role` plus accessible `name`, limited to `within`; the best-scored candidate wins, or the `nth` match when set. An element that stays outside the viewport after scrolling it into view fails with `target_not_found`.
Any of:
Option 1unknown
Option 2unknown
Option 3unknown
timeout_ms integer
Option 4
kind string REQUIRED
Enum: type
target unknown | unknown | unknown
How an action finds its element (the same rules as render timelines, `timeline.schema.json#/$defs/target`): candidates match `selector` (CSS), `text` (visible text, case-insensitive substring unless `exact`) or `role` plus accessible `name`, limited to `within`; the best-scored candidate wins, or the `nth` match when set. An element that stays outside the viewport after scrolling it into view fails with `target_not_found`.
Any of:
Option 1unknown
Option 2unknown
Option 3unknown
text string REQUIRED
clear boolean
Clear the field before typing.
interval_ms integer
Delay between characters.
timeout_ms integer
Option 5
kind string REQUIRED
Enum: key
key string REQUIRED
`KeyboardEvent.key`, such as `Enter`, `Tab` or `a`.
modifiers string[]
Modifier keys held during the action.
Array of: string
Option 6
kind string REQUIRED
Enum: scroll
target unknown | unknown | unknown
How an action finds its element (the same rules as render timelines, `timeline.schema.json#/$defs/target`): candidates match `selector` (CSS), `text` (visible text, case-insensitive substring unless `exact`) or `role` plus accessible `name`, limited to `within`; the best-scored candidate wins, or the `nth` match when set. An element that stays outside the viewport after scrolling it into view fails with `target_not_found`.
Any of:
Option 1unknown
Option 2unknown
Option 3unknown
to object
A position in CSS pixels.
x number REQUIRED
y number REQUIRED
by object
A position in CSS pixels.
x number REQUIRED
y number REQUIRED
Option 7
kind string REQUIRED
Enum: select
target unknown | unknown | unknown REQUIRED
How an action finds its element (the same rules as render timelines, `timeline.schema.json#/$defs/target`): candidates match `selector` (CSS), `text` (visible text, case-insensitive substring unless `exact`) or `role` plus accessible `name`, limited to `within`; the best-scored candidate wins, or the `nth` match when set. An element that stays outside the viewport after scrolling it into view fails with `target_not_found`.
Any of:
Option 1unknown
Option 2unknown
Option 3unknown
value string REQUIRED
The option value to select.
Option 8
kind string REQUIRED
Enum: wait
ms integer
for object
Exactly one condition to wait for (the same as a timeline `wait` action's `for`).
selector string
An element matching this CSS selector is visible.
text string
This text is visible.
url string
The page URL contains this substring.
network_idle boolean
The network has been quiet.
Enum: true
timeout_ms integer
Option 9
kind string REQUIRED
Enum: evaluate
expression string REQUIRED
JavaScript evaluated in the page; a returned promise is awaited and the JSON-serializable result is returned.
Option 10
kind string REQUIRED
Enum: screenshot
target unknown | unknown | unknown
How an action finds its element (the same rules as render timelines, `timeline.schema.json#/$defs/target`): candidates match `selector` (CSS), `text` (visible text, case-insensitive substring unless `exact`) or `role` plus accessible `name`, limited to `within`; the best-scored candidate wins, or the `nth` match when set. An element that stays outside the viewport after scrolling it into view fails with `target_not_found`.
Any of:
Option 1unknown
Option 2unknown
Option 3unknown
full_page boolean
format string
Enum: jpeg, png, webp
quality integer
Option 11
kind string REQUIRED
Enum: markdown
only_main_content boolean
target_id string
The page to act on (from `pages`); defaults to the most recently active page.
continue_on_error boolean
timeout_ms integer

Responses

200 One result per action, in order.
application/json
object string REQUIRED
Enum: act_result
session_id string REQUIRED
ok boolean REQUIRED
`true` when every action succeeded.
results object[] REQUIRED
Array of:
kind string REQUIRED
Enum: navigate, click, hover, type, key, scroll, select, wait, evaluate, screenshot, markdown
status string REQUIRED
Enum: ok, failed, skipped
duration_ms number REQUIRED
url string
`navigate`: the URL after redirects.
status_code integer
`navigate`: the HTTP status.
value unknown
`evaluate`: the JSON result.
screenshot object
format string REQUIRED
Enum: jpeg, png, webp
width integer REQUIRED
height integer REQUIRED
data_base64 string (byte) REQUIRED
markdown string
`markdown`: the page as Markdown.
error object
An RFC 9457 problem. `code` is stable and listed in the errors catalog; `type` links to its entry. `detail` is human-readable and may change. Extension members appear only for the codes that document them; ignore members you do not recognize.
type string (uri) REQUIRED
`https://docs.beta.b3os.org/errors#<code>`.
title string REQUIRED
Short summary of the problem type.
status integer REQUIRED
The HTTP status code, repeated for convenience.
code string REQUIRED
Stable machine-readable error code from the errors catalog.
detail string REQUIRED
Explanation of this occurrence. Never contains secrets.
instance string
The request ID (`req_…`) of the request that failed. Include it when you contact support.
errors object[]
Validation problems, one per field (`invalid_request` and the other codes that list `errors`).
Array of:
pointer string REQUIRED
Where the problem is: a JSON pointer into the request body (`/url`, `/steps/0/uses`), or `#query/<name>`, `#path/<name>` or `#header/<name>` for parameters.
detail string REQUIRED
What is wrong there.
reason string
Finer-grained cause for codes that document one (`unauthorized`, `forbidden`, `egress_denied`, `upstream_unreachable`, …).
required_scope string
The scope that was missing (`forbidden` with `reason: scope`).
retry_after_s integer
Same value as the `Retry-After` header, for clients that only read the body.
limit object
The limit an error refers to. IDs and current values are listed by `GET /v1/limits`.
id string REQUIRED
Limit ID, such as `vision.scrape.rpm` or `sandboxes.concurrent`.
limit number REQUIRED
The limit's value.
unit string REQUIRED
What the limit counts, such as `requests`, `sandboxes`, `vcpu` or `MiB`.
window_s integer
Window length in seconds for rate limits and quotas.
current number
Current usage against the limit, when known.
current_status string
The object's current status, for state conflicts such as `sandbox_not_running`.
upstream_status integer
The HTTP status the upstream returned (`upstream_error`, `action_output_invalid`).
step string
The failed step, as a step ID (routines), an index (`act`) or a path such as `beats/2/do/0` (render).
pack_error object
The error a pack's error mapping chose (`upstream_error` from pack actions).
code string REQUIRED
message string REQUIRED
retryable boolean REQUIRED
timing object REQUIRED
Server-side phase durations in milliseconds with microsecond resolution: the same phases as the `Server-Timing` header, keyed `<phase>_ms`. `total_ms` is always present. Phase names are product-specific and documented per operation. Example values in this reference are illustrative, not measurements.
total_ms number REQUIRED
400 The request failed validation (`invalid_request`, `invalid_url`).
401 Missing, malformed, expired or revoked credentials (`unauthorized`).
403 Authenticated but not allowed (`forbidden`, `egress_denied`).
404 No such browser session in this project (`session_not_found`).
409 The session already closed (`session_closed`).
413 The request body exceeds this operation's limit.
415 The Content-Type is not accepted by this operation.
429 A rate, concurrency or quota limit applies (`rate_limited`, `concurrency_limited`, `quota_exceeded`).
500 Something failed inside B3OS (`internal`). Retry with backoff.
503 The node or the product is not available right now (`node_unavailable`, `product_unavailable`, `capacity_unavailable`). 
curl -X POST 'https://beta-api.b3os.org/v1/vision/sessions/brs_01m5t3y30wcksxf22t9gwk9fwb/act' \  -H 'Authorization: Bearer YOUR_API_TOKEN' \  -H 'Content-Type: application/json' \  -d '{  "actions": [    {      "kind": "navigate",      "url": "https://app.example.com/login"    },    {      "kind": "type",      "target": {        "selector": "#email"      },      "text": "agent@example.com"    },    {      "kind": "click",      "target": {        "role": "button",        "name": "Continue"      }    },    {      "kind": "wait",      "for": {        "selector": "[data-test=\"dashboard\"]"      }    },    {      "kind": "markdown"    }  ]}'
const response = await fetch('https://beta-api.b3os.org/v1/vision/sessions/brs_01m5t3y30wcksxf22t9gwk9fwb/act', {  method: 'POST',  headers: {      "Authorization": "Bearer YOUR_API_TOKEN",      "Content-Type": "application/json"  },  body: JSON.stringify({    "actions": [      {        "kind": "navigate",        "url": "https://app.example.com/login"      },      {        "kind": "type",        "target": {          "selector": "#email"        },        "text": "agent@example.com"      },      {        "kind": "click",        "target": {          "role": "button",          "name": "Continue"        }      },      {        "kind": "wait",        "for": {          "selector": "[data-test=\"dashboard\"]"        }      },      {        "kind": "markdown"      }    ]  })});const data = await response.json();console.log(data);
import requestsheaders = {    'Authorization': 'Bearer YOUR_API_TOKEN'}response = requests.post('https://beta-api.b3os.org/v1/vision/sessions/brs_01m5t3y30wcksxf22t9gwk9fwb/act', headers=headers, json={  "actions": [    {      "kind": "navigate",      "url": "https://app.example.com/login"    },    {      "kind": "type",      "target": {        "selector": "#email"      },      "text": "agent@example.com"    },    {      "kind": "click",      "target": {        "role": "button",        "name": "Continue"      }    },    {      "kind": "wait",      "for": {        "selector": "[data-test=\"dashboard\"]"      }    },    {      "kind": "markdown"    }  ]})print(response.json())
200 Response 
{  "object": "act_result",  "session_id": "brs_01m5t3y30wcksxf22t9gwk9fwb",  "ok": true,  "results": [    {      "kind": "navigate",      "status": "ok",      "duration_ms": 212.4,      "url": "https://app.example.com/login",      "status_code": 200    },    {      "kind": "type",      "status": "ok",      "duration_ms": 18    },    {      "kind": "click",      "status": "ok",      "duration_ms": 6.2    },    {      "kind": "wait",      "status": "ok",      "duration_ms": 310.9    },    {      "kind": "markdown",      "status": "ok",      "duration_ms": 4.1,      "markdown": "# Dashboard"    }  ],  "timing": {    "actions_ms": 551.6,    "total_ms": 552.3  }}
Previous Close a browser session
Next Connect over CDP
Ask a question... ⌘I