> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape a page with curl

Turn a URL into clean Markdown with one request to B3OS vision, then read the Server-Timing header to see where the time went.

You need a key in `B3OS_API_KEY` ([get one](/quickstart#get-a-key)) and `curl`.

**Scrape a page**

Send a URL and the formats you want back:

    ```bash
    curl -s https://beta-api.b3os.org/v1/vision/scrape \
      -H "Authorization: Bearer $B3OS_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://example.com", "formats": ["markdown", "links"]}'
    ```
  **Read the response**

The page comes back as Markdown, with what served it and how long each phase took. Abridged:

    ```jsonc
    {
      "object": "scrape",
      "url": "https://example.com",
      "status_code": 200,               // the site's status; a 404 page is still a successful scrape
      "tier": "fetch",                  // cache, fetch or browser
      "cache": { "state": "miss", "age_s": 0 },
      "markdown": "# Example Domain\n\n…",
      "links": ["…"],
      "warnings": [],
      "timing": { "fetch_ms": …, "extract_ms": …, "total_ms": … }
    }
    ```

    `tier` says what served the page: `cache`, `fetch` (plain HTTP) or `browser` (a warm Chrome, used when the page needs JavaScript). `cache.state` is `hit`, `miss` or `bypass`. A scrape of the same URL within two days can come from the cache, and `timing` holds the server-side phase durations in milliseconds.
  **See where the time went**

Every response carries a `Server-Timing` header and an `x-request-id`. Print the headers instead of the body:

    ```bash
    curl -s -o /dev/null -D - https://beta-api.b3os.org/v1/vision/scrape \
      -H "Authorization: Bearer $B3OS_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://example.com"}' | grep -i -E 'server-timing|x-request-id'
    ```

    The edge's phases (`edge_auth`, `edge_upstream`) come first, then the node's (`fetch`, `extract`, `total`). Copy the `x-request-id` value and ask the request log for the full record:

    ```bash
    export REQUEST_ID="req_..."   # the x-request-id value from the headers
    curl -s "https://beta-api.b3os.org/v1/requests/$REQUEST_ID" \
      -H "Authorization: Bearer $B3OS_API_KEY"
    ```

    [Timing](/concepts/timing) explains each phase.
  **Read a URL in one line**

Put the target URL after `/v1/vision/r/` and the page comes back as Markdown:

    ```bash
    curl -s https://beta-api.b3os.org/v1/vision/r/https://example.com \
      -H "Authorization: Bearer $B3OS_API_KEY"
    ```

    Options go in headers so the URL stays untouched: `X-B3OS-Format` picks `html`, `text` or `json` instead, and `X-B3OS-Engine: browser` forces a browser.

## Next

- [Scrape guide](/vision/scrape): formats, main-content extraction, browser actions, profiles, caching.
- [Crawl and batch](/vision/crawl) for many pages at once.
- [Firecrawl compatibility](/compat/firecrawl) if you already use a Firecrawl SDK: change the base URL and key.