> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Vision

Turn web pages into Markdown, map and crawl sites, and drive dedicated browsers over CDP, from browser pools that are already running on the node.

Vision is how your agent reads and uses the web. It turns pages into clean Markdown and other formats, discovers and crawls sites, and gives your agent a browser of its own to click through, all from browsers and contexts that are already running when the request arrives.

## What to call

| You want | Call | Bills |
|---|---|---|
| One page as Markdown, HTML, text, links, metadata or a screenshot | [`POST /v1/vision/scrape`](/vision/scrape), or `GET /v1/vision/r/<url>` for a one-line read (Markdown by default) | One `vision.page` |
| A site's URLs, from its sitemaps and links | [`POST /v1/vision/map`](/vision/crawl#map-a-site) | One `vision.page` |
| Up to 1,000 URLs you already have | [`POST /v1/vision/batches`](/vision/crawl#scrape-a-list-of-urls) | One `vision.page` per page |
| A site, following its links | [`POST /v1/vision/crawls`](/vision/crawl#crawl-a-site) | One `vision.page` per page |
| A browser your agent drives, with clicks, forms and CDP | [Sessions](/vision/sessions) | `vision.browser_ms` while open |
| To stay signed in across sessions and scrapes | [Profiles](/vision/profiles) | Nothing |

## How a scrape is served

A scrape has three ways to get a page, and `engine: "auto"`, the default, takes the cheapest one that works:

1. **Cache.** A copy no older than `max_age_ms` (two days by default) answers at once.
2. **Fetch.** A plain HTTP request, with the page converted to Markdown in Rust inside the B3OS daemon.
3. **Browser.** A warm Chrome page from a pool of pre-created contexts, used only when the page needs JavaScript to show its content.

`engine: "fetch"` never starts a browser. `engine: "browser"` always renders, and browser actions, screenshots and profiles imply it. The response's `tier` says which one served the page; each bills the same single `vision.page`.

## The cache

A page enters the shared cache only when the request had no custom headers or cookies, no profile, session or browser actions, the default location, and the site set no cookies. Anything else is cached for your organization only, or not cached at all. `cache.state` in the response is `hit`, `miss` (fetched and stored) or `bypass` (fetched and not shared), and `cache.age_s` is the copy's age. Cached pages are kept for two days. Set `max_age_ms: 0` to always fetch, or `store_in_cache: false` to keep a result out.

## Browsers

Two kinds of browser serve vision, and they never mix:

- **The scrape pool** serves scrapes, batches and crawls that need JavaScript. Its browsers are driven only by B3OS; nobody connects to them over CDP.
- **Session browsers** are whole browser processes, one per session with its own profile directory, claimed from a warm pool. Your agent drives one with batched actions or any CDP client, through a proxy that exposes only that session's pages. During the beta Chrome runs without its renderer sandbox; the [beta contract](/concepts/beta#isolation) says what that means.

Both are Chrome headless shell by default. All their traffic goes through the egress guard.

## Rules vision follows

- **robots.txt and Content Signals** are honored by default. A URL they disallow fails with [`robots_disallowed`](/errors#robots_disallowed); the beta has no override.
- **Politeness.** Crawls space their requests to one host.
- **Egress.** Every fetch and navigation goes through the egress guard, so a URL that resolves to a private, loopback or metadata address fails with [`egress_denied`](/errors#egress_denied).
- **Ads and trackers** are blocked by default in scrapes (`block_ads`), and images, media and fonts are not downloaded unless you ask for a screenshot.

## Scopes

| Scope | For |
|---|---|
| `vision:run` | Scrape, map, crawl, batch scrape, and drive sessions (`act`, CDP) |
| `vision:write` | Create and close sessions; create, change and delete profiles; export a profile's state |
| `vision:read` | List and read sessions, profiles, crawls and batches |

## Guides

**[Scrape a page](/vision/scrape)**

Formats, main content, browser actions, screenshots and the cache.
  **[Map, batch and crawl](/vision/crawl)**

Many pages as one job, with events and webhooks.
  **[Browser sessions](/vision/sessions)**

A dedicated browser: batched actions or Playwright and Puppeteer over CDP.
  **[Profiles](/vision/profiles)**

Saved cookies and storage, so agents start signed in.

[Limits and FAQ](/vision/limits) lists the caps and the common questions. Firecrawl users can keep their SDK: see [Firecrawl compatibility](/compat/firecrawl).