List a crawl's page results

Lists crawled pages in completion order, including failures. Available while the crawl runs.

GET /v1/vision/crawls/{crawl_id}/pages
Lists crawled pages in completion order, including failures. Available while the crawl runs.

Authentication

Bearer Token (b3os_sk_ API key or b3os_pat_ token) API Key (cookie: __Host-b3os_session)

Path Parameters

crawl_id string required path
The crawl ID (`crawl_…`).
Example: "crawl_01m5t4gh4qhgs70zwajezbe5p0"

Query Parameters

limit integer optional query
Page size, from 1 to 100.
cursor string optional query
Opaque cursor from the previous page's `next_cursor`. Omit it for the first page.
status string optional query
Only pages that succeeded, or only pages that failed.

Responses

200 A page of results.
application/json
object string REQUIRED
Enum: list
data object[] REQUIRED
Array of:
object string REQUIRED
Enum: page_result
url string REQUIRED
status string REQUIRED
Enum: succeeded, failed
depth integer
Link hops from the crawl's start URL (crawls only).
scraped_at string (date-time)
scrape object
object string REQUIRED
Enum: scrape
url string (uri) REQUIRED
The requested URL.
final_url string (uri) REQUIRED
The URL after redirects.
status_code integer REQUIRED
The HTTP status the site returned. A 404 page is still a successful scrape.
content_type string
tier string REQUIRED
What served the page.
Enum: cache, fetch, browser
cache object REQUIRED
state string REQUIRED
`hit` served from cache; `miss` fetched and stored; `bypass` fetched and not shared (custom headers, profile, actions).
Enum: hit, miss, bypass
age_s integer REQUIRED
Age of the served copy in seconds.
markdown string
html string
raw_html string
text string
links string[]
Array of: string
screenshot object
format string REQUIRED
Enum: jpeg, png, webp
width integer REQUIRED
height integer REQUIRED
data_base64 string (byte) REQUIRED
json unknown
Data matching `json.schema`, when the `json` format was requested.
metadata object
Metadata read from the page.
title string
description string
language string
canonical_url string (uri)
favicon string (uri)
og object
Open Graph properties, keyed without the `og:` prefix.
published_time string
modified_time string
robots string
The page's robots meta tag.
actions object[]
Results of `actions`, in order.
Array of:
kind string REQUIRED
Enum: navigate, click, hover, type, key, scroll, select, wait, evaluate, screenshot, markdown
status string REQUIRED
Enum: ok, failed, skipped
duration_ms number REQUIRED
url string
`navigate`: the URL after redirects.
status_code integer
`navigate`: the HTTP status.
value unknown
`evaluate`: the JSON result.
screenshot object
format string REQUIRED
Enum: jpeg, png, webp
width integer REQUIRED
height integer REQUIRED
data_base64 string (byte) REQUIRED
markdown string
`markdown`: the page as Markdown.
error object
An RFC 9457 problem. `code` is stable and listed in the errors catalog; `type` links to its entry. `detail` is human-readable and may change. Extension members appear only for the codes that document them; ignore members you do not recognize.
type string (uri) REQUIRED
`https://docs.beta.b3os.org/errors#<code>`.
title string REQUIRED
Short summary of the problem type.
status integer REQUIRED
The HTTP status code, repeated for convenience.
code string REQUIRED
Stable machine-readable error code from the errors catalog.
detail string REQUIRED
Explanation of this occurrence. Never contains secrets.
instance string
The request ID (`req_…`) of the request that failed. Include it when you contact support.
errors object[]
Validation problems, one per field (`invalid_request` and the other codes that list `errors`).
...
reason string
Finer-grained cause for codes that document one (`unauthorized`, `forbidden`, `egress_denied`, `upstream_unreachable`, …).
required_scope string
The scope that was missing (`forbidden` with `reason: scope`).
retry_after_s integer
Same value as the `Retry-After` header, for clients that only read the body.
limit object
The limit an error refers to. IDs and current values are listed by `GET /v1/limits`.
...
current_status string
The object's current status, for state conflicts such as `sandbox_not_running`.
upstream_status integer
The HTTP status the upstream returned (`upstream_error`, `action_output_invalid`).
step string
The failed step, as a step ID (routines), an index (`act`) or a path such as `beats/2/do/0` (render).
pack_error object
The error a pack's error mapping chose (`upstream_error` from pack actions).
...
warnings string[] REQUIRED
Non-fatal problems, such as a format that could not be produced.
Array of: string
timing object REQUIRED
Server-side phase durations in milliseconds with microsecond resolution: the same phases as the `Server-Timing` header, keyed `<phase>_ms`. `total_ms` is always present. Phase names are product-specific and documented per operation. Example values in this reference are illustrative, not measurements.
total_ms number REQUIRED
error object
An RFC 9457 problem. `code` is stable and listed in the errors catalog; `type` links to its entry. `detail` is human-readable and may change. Extension members appear only for the codes that document them; ignore members you do not recognize.
type string (uri) REQUIRED
`https://docs.beta.b3os.org/errors#<code>`.
title string REQUIRED
Short summary of the problem type.
status integer REQUIRED
The HTTP status code, repeated for convenience.
code string REQUIRED
Stable machine-readable error code from the errors catalog.
detail string REQUIRED
Explanation of this occurrence. Never contains secrets.
instance string
The request ID (`req_…`) of the request that failed. Include it when you contact support.
errors object[]
Validation problems, one per field (`invalid_request` and the other codes that list `errors`).
Array of:
pointer string REQUIRED
Where the problem is: a JSON pointer into the request body (`/url`, `/steps/0/uses`), or `#query/<name>`, `#path/<name>` or `#header/<name>` for parameters.
detail string REQUIRED
What is wrong there.
reason string
Finer-grained cause for codes that document one (`unauthorized`, `forbidden`, `egress_denied`, `upstream_unreachable`, …).
required_scope string
The scope that was missing (`forbidden` with `reason: scope`).
retry_after_s integer
Same value as the `Retry-After` header, for clients that only read the body.
limit object
The limit an error refers to. IDs and current values are listed by `GET /v1/limits`.
id string REQUIRED
Limit ID, such as `vision.scrape.rpm` or `sandboxes.concurrent`.
limit number REQUIRED
The limit's value.
unit string REQUIRED
What the limit counts, such as `requests`, `sandboxes`, `vcpu` or `MiB`.
window_s integer
Window length in seconds for rate limits and quotas.
current number
Current usage against the limit, when known.
current_status string
The object's current status, for state conflicts such as `sandbox_not_running`.
upstream_status integer
The HTTP status the upstream returned (`upstream_error`, `action_output_invalid`).
step string
The failed step, as a step ID (routines), an index (`act`) or a path such as `beats/2/do/0` (render).
pack_error object
The error a pack's error mapping chose (`upstream_error` from pack actions).
code string REQUIRED
message string REQUIRED
retryable boolean REQUIRED
next_cursor string | null REQUIRED
400 The request failed validation (`invalid_request`, `invalid_url`).
401 Missing, malformed, expired or revoked credentials (`unauthorized`).
403 Authenticated but not allowed (`forbidden`, `egress_denied`).
404 No such crawl in this project (`crawl_not_found`).
429 A rate, concurrency or quota limit applies (`rate_limited`, `concurrency_limited`, `quota_exceeded`).
500 Something failed inside B3OS (`internal`). Retry with backoff.
503 The node or the product is not available right now (`node_unavailable`, `product_unavailable`, `capacity_unavailable`). 
curl -X GET 'https://beta-api.b3os.org/v1/vision/crawls/crawl_01m5t4gh4qhgs70zwajezbe5p0/pages' \  -H 'Authorization: Bearer YOUR_API_TOKEN'
const response = await fetch('https://beta-api.b3os.org/v1/vision/crawls/crawl_01m5t4gh4qhgs70zwajezbe5p0/pages', {  method: 'GET',  headers: {      "Authorization": "Bearer YOUR_API_TOKEN"  }});const data = await response.json();console.log(data);
import requestsheaders = {    'Authorization': 'Bearer YOUR_API_TOKEN'}response = requests.get('https://beta-api.b3os.org/v1/vision/crawls/crawl_01m5t4gh4qhgs70zwajezbe5p0/pages', headers=headers)print(response.json())
200 Response 
{  "object": "list",  "data": [    {      "object": "page_result",      "url": "https://docs.example.com/guides/start",      "status": "failed",      "depth": 1,      "scraped_at": "2026-10-25T18:10:09.551Z",      "error": {        "type": "https://docs.beta.b3os.org/errors#navigation_failed",        "title": "Navigation failed",        "status": 502,        "code": "navigation_failed",        "detail": "net::ERR_CONNECTION_RESET",        "reason": "net::ERR_CONNECTION_RESET"      }    }  ],  "next_cursor": null}
Previous Get a crawl
Next Stream a crawl's events
Ask a question... ⌘I