Start a batch scrape (Firecrawl)

Firecrawl's `POST /v1/batch/scrape`, served by a vision batch scrape (`batch_` job). The scrape options are at the top level of the body, as Firecrawl takes them. `webhook` deliveries use Firecrawl's payload and `X-Firecrawl-Signature`. Each page bills one `vision.page`.

POST /compat/firecrawl/v1/batch/scrape
Firecrawl's `POST /v1/batch/scrape`, served by a vision batch scrape (`batch_` job). The scrape options are at the top level of the body, as Firecrawl takes them. `webhook` deliveries use Firecrawl's payload and `X-Firecrawl-Signature`. Each page bills one `vision.page`.

Authentication

Bearer Token (b3os_sk_ API key or b3os_pat_ token)

Header Parameters

x-idempotency-key string optional header
Idempotency key as the Firecrawl SDKs send it; same semantics as `Idempotency-Key`.

Request Body required

application/json
urls string (uri)[] REQUIRED
Array of: string (uri)
webhook string (uri) | object
A webhook URL, or an object with the URL and options.
One of:
Option 1string (uri)
Option 2
url string (uri) REQUIRED
headers object
metadata object
events string[]
Array of: string
maxConcurrency integer
ignoreInvalidURLs boolean
appendToId string
Not supported by B3OS: setting it returns `400` with code `unsupported_option`.
zeroDataRetention boolean
Do not cache or store the result.
integration string
Accepted and ignored (the Firecrawl SDKs send it).
origin string
Accepted and ignored (the Firecrawl SDKs send it).
formats string[]
`changeTracking` returns `unsupported_option`.
Array of: string
onlyMainContent boolean
includeTags string[]
Array of: string
excludeTags string[]
Array of: string
maxAge integer
Accept a cached copy up to this old, in milliseconds. Default 0 (always fetch).
headers object
waitFor integer
mobile boolean
skipTlsVerification boolean
timeout integer
Milliseconds; values above 60000 are treated as 60000.
parsePDF boolean
PDF parsing is not supported; PDFs return `content_type_unsupported`.
jsonOptions object
schema object
systemPrompt string
prompt string
checkPromptInjection boolean
Not supported by B3OS: setting it returns `400` with code `unsupported_option`.
actions (object | object | object | object | object | object | object | object | object)[]
Array of:
One of:
Option 1
type string REQUIRED
Enum: wait
milliseconds integer
selector string
Option 2
type string REQUIRED
Enum: screenshot
fullPage boolean
quality integer
viewport object
width integer REQUIRED
height integer REQUIRED
Option 3
type string REQUIRED
Enum: click
selector string REQUIRED
all boolean
Option 4
type string REQUIRED
Enum: write
text string REQUIRED
Option 5
type string REQUIRED
Enum: press
key string REQUIRED
Option 6
type string REQUIRED
Enum: scroll
direction string
Enum: up, down
selector string
Option 7
type string REQUIRED
Enum: scrape
Option 8
type string REQUIRED
Enum: executeJavascript
script string REQUIRED
Option 9
type string REQUIRED
Enum: pdf
format string
landscape boolean
scale number
location object
country string
ISO 3166-1 alpha-2 code. B3OS emulates the language and time zone; it has no per-country proxies.
languages string[]
Array of: string
removeBase64Images boolean
blockAds boolean
proxy string
`enhanced` returns `unsupported_option`.
Enum: basic, enhanced, auto
storeInCache boolean
changeTrackingOptions object
Not supported by B3OS: setting it returns `400` with code `unsupported_option`.
threatProtection object
Not supported by B3OS: setting it returns `400` with code `unsupported_option`.

Responses

200 The batch was started.
application/json
success boolean REQUIRED
Enum: true
id string REQUIRED
url string (uri) REQUIRED
The job's status URL.
invalidURLs string[] | null
Array of: string
400 The body failed validation, uses an unsupported option, or a URL is invalid (`invalid_request`, `unsupported_option`, `invalid_url`).
401 Missing or invalid B3OS key (`unauthorized`). Firecrawl `fc-` keys are not accepted.
402 No credits left or the key's spend cap is reached (`credits_exhausted`, `spend_cap_exceeded`).
403 The URL is refused by the egress guard or robots.txt, or the key lacks a scope (`egress_denied`, `robots_disallowed`, `forbidden`).
429 A rate or concurrency limit applies (`rate_limited`, `concurrency_limited`).
500 Something failed inside B3OS (`internal`).
503 The node or vision is unavailable (`node_unavailable`, `product_unavailable`). 
curl -X POST 'https://beta-api.b3os.org/compat/firecrawl/v1/batch/scrape' \  -H 'Authorization: Bearer YOUR_API_TOKEN' \  -H 'Content-Type: application/json' \  -d '{  "urls": [    "https://example.com/blog/a",    "https://example.com/blog/b"  ],  "formats": [    "markdown"  ]}'
const response = await fetch('https://beta-api.b3os.org/compat/firecrawl/v1/batch/scrape', {  method: 'POST',  headers: {      "Authorization": "Bearer YOUR_API_TOKEN",      "Content-Type": "application/json"  },  body: JSON.stringify({    "urls": [      "https://example.com/blog/a",      "https://example.com/blog/b"    ],    "formats": [      "markdown"    ]  })});const data = await response.json();console.log(data);
import requestsheaders = {    'Authorization': 'Bearer YOUR_API_TOKEN'}response = requests.post('https://beta-api.b3os.org/compat/firecrawl/v1/batch/scrape', headers=headers, json={  "urls": [    "https://example.com/blog/a",    "https://example.com/blog/b"  ],  "formats": [    "markdown"  ]})print(response.json())
200 Response 
{  "success": true,  "id": "batch_01m5t3v176vrhf8a1mn570c5gd",  "url": "https://beta-api.b3os.org/compat/firecrawl/v1/batch/scrape/batch_01m5t3v176vrhf8a1mn570c5gd",  "invalidURLs": []}
Previous Scrape a URL (Firecrawl)
Next Get a batch scrape (Firecrawl)
Ask a question... ⌘I