> ## Documentation Index
> Fetch the complete documentation index at: https://docs.beta.b3os.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Jobs, streams and webhooks

The one contract every long-running B3OS operation follows, how to wait, stream or get a webhook, and how to retry writes safely.

Crawls, batch scrapes, videos, routine runs and extractions take longer than a request should. They are **jobs**, and every job works the same way.

## The job object

```json
{
  "id": "crawl_01m5t4gh4qhgs70zwajezbe5p0",
  "object": "crawl",
  "status": "running",
  "created_at": "2026-10-25T18:10:02.114Z",
  "updated_at": "2026-10-25T18:10:20.402Z",
  "started_at": "2026-10-25T18:10:02.120Z",
  "finished_at": null,
  "progress": { "completed": 12, "failed": 0, "total": 37, "unit": "pages" },
  "expires_at": null,
  "metadata": {}
}
```

`status` moves from `queued` to `running` to one of `succeeded`, `failed` or `canceled`, and never leaves a terminal status. `progress` counts in the job's own unit (pages, phases, steps). Once the job ends, `result` holds the output on success, `error` holds a [problem](/errors) on failure or cancel, `timing` has the final phase durations, and `expires_at` says when the results are deleted.

## Three ways to get the result

**Wait in the request.** Add `?wait=<seconds>` (up to 60) to the `POST` that creates the job. If it finishes in time you get `200` with the finished job. If not, you get `202` with the job as it is, a `Location` header to poll and a `Retry-After` header.

```bash
curl -s "https://beta-api.b3os.org/v1/vision/crawls?wait=30" \
  -H "Authorization: Bearer $B3OS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://docs.example.com", "limit": 20}'
```

**Stream its events.** Send `Accept: text/event-stream` on the creating `POST`, or open `GET .../{id}/events` later.

**Get a webhook.** Pass `webhook: {url, events}` when you create the job and B3OS posts the events to your URL. See [webhooks](#webhooks) below.

You can also poll `GET` on the job with its ID. To stop a job, `POST .../{id}/cancel`: it ends `canceled` with the code [`job_canceled`](/errors#job_canceled), and work already done stays done and billed.

A request that does not stream gets its response within 65 seconds: no handler works on one request for more than 60 seconds, `?wait` included, and the rest covers queueing and the edge. Anything longer is a job. Streams stay open longer, up to each operation's own limit: an hour for job event streams, streamed commands and sandbox terminals, 30 minutes for a browser session's CDP socket, 10 minutes for a streamed chat completion. Reconnect an event stream with `Last-Event-ID` to continue past it.

## Event streams

Every stream in B3OS (job events, command output, sandbox logs) uses the same server-sent events framing:

```text
id: 14
event: page.succeeded
data: {"type":"page.succeeded","ts":"2026-10-25T18:10:20.402Z","job_id":"crawl_01m5t4gh4qhgs70zwajezbe5p0","seq":14,"page":{"object":"page_result","url":"https://docs.example.com/guides/keys","status":"succeeded","depth":2}}
```

- `seq` goes up by one per event and equals the SSE `id`. If the connection drops, reconnect with `Last-Event-ID: <seq>` and the stream resumes after that event.
- A `: ping` comment arrives every 15 seconds, so idle proxies keep the connection open.
- Lifecycle events are `job.queued`, `job.running`, `job.progress` and the terminal `job.succeeded`, `job.failed` and `job.canceled`. The terminal event carries the final job, timing included, and ends the stream.
- Products add their own events, such as `page.succeeded` for crawls, `step.waiting` for routine runs and `video.artifact` for videos. Each operation in the [API reference](/api/overview) lists the events it sends.

```bash
curl -N -s "https://beta-api.b3os.org/v1/vision/crawls/$CRAWL_ID/events" \
  -H "Authorization: Bearer $B3OS_API_KEY"
```

## Webhooks

Pass a webhook when you create a job:

```json
{ "url": "https://docs.example.com", "webhook": { "url": "https://example.com/hooks/b3os", "events": ["job.succeeded", "job.failed"] } }
```

`events` defaults to the terminal events; `["*"]` sends every event. B3OS posts each one as a JSON event (the job is abridged here):

```json
{
  "id": "evt_01m5t4p5ycda35cq27epnbfqhe",
  "object": "event",
  "type": "job.succeeded",
  "created_at": "2026-10-25T18:10:41.030Z",
  "job_id": "crawl_01m5t4gh4qhgs70zwajezbe5p0",
  "data": { "job": { "id": "crawl_01m5t4gh4qhgs70zwajezbe5p0", "object": "crawl", "status": "succeeded" } }
}
```

The event `id` stays the same across retries of one delivery, so use it to drop duplicates. Failed deliveries are retried with backoff for 24 hours. Deliveries leave from the node through the egress guard, so URLs on private or loopback addresses are refused.

### Verify the signature

Every delivery carries `B3OS-Signature: t=<unix seconds>,v1=<hex>`. The signature is HMAC-SHA256 over `<t>.<raw body>`, keyed with the signing secret of your URL: the UTF-8 bytes of the whole `whsec_...` string, prefix included, never decoded. Each attempt is signed when it is sent, so a retry hours later carries a fresh `t` and still passes a five-minute check. Get the secret once, when you set up the receiver; the same URL always gets the same secret. The call needs a key with `platform:write`, which the first key you get with your invite does not have: create one in the [dashboard](https://beta.b3os.org/app) or with [`POST /v1/keys`](/api/post-v1-keys) first.

```bash
curl -s https://beta-api.b3os.org/v1/webhooks/secret \
  -H "Authorization: Bearer $B3OS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/hooks/b3os"}'
# {"object": "webhook_secret", "url": "https://example.com/hooks/b3os", "secret": "whsec_...", ...}
```

Then check every delivery against the raw body, before parsing it. The SDKs do it in one call and throw `WebhookVerificationError` for a delivery that fails; the versions without the SDK return `false` for anything malformed instead of throwing:

**TypeScript**

```ts
    import { verifyWebhook, WebhookVerificationError } from "@b3os/sdk/webhooks";

    // rawBody is the request body exactly as received, before any JSON parsing
    try {
      const event = await verifyWebhook(rawBody, request.headers.get("b3os-signature"), process.env.B3OS_WEBHOOK_SECRET ?? "");
      console.log(event.type, event.id);
    } catch (error) {
      if (!(error instanceof WebhookVerificationError)) throw error;
      // answer 400 and drop the delivery; error.reason says why
    }
    ```
  **Python**

```python
    import os
    from b3os import WebhookVerificationError, verify_webhook

    # raw_body is the request body exactly as received, before any JSON parsing
    try:
        event = verify_webhook(raw_body, headers.get("b3os-signature"), os.environ["B3OS_WEBHOOK_SECRET"])
        print(event["type"], event["id"])
    except WebhookVerificationError as error:
        print("rejected:", error.reason)  # answer 400 and drop the delivery
    ```
  **Node, no SDK**

```ts
    import { createHmac, timingSafeEqual } from "node:crypto";

    // True only for a delivery signed with `secret` less than five minutes ago. Never throws.
    export function verifyB3osSignature(
      rawBody: string | Buffer,
      header: string | null | undefined,
      secret: string,
    ): boolean {
      if (typeof header !== "string") return false;
      let t: number | undefined;
      const signatures: Buffer[] = [];
      for (const part of header.split(",")) {
        const eq = part.indexOf("=");
        if (eq < 0) continue;
        const key = part.slice(0, eq).trim();
        const value = part.slice(eq + 1).trim();
        if (key === "t" && /^[0-9]{1,12}$/.test(value)) t = Number(value);
        else if (key === "v1" && /^[0-9a-fA-F]{64}$/.test(value)) signatures.push(Buffer.from(value, "hex"));
      }
      if (t === undefined || Math.abs(Date.now() / 1000 - t) > 300) return false;
      const expected = createHmac("sha256", secret).update(`${t}.`).update(rawBody).digest();
      return signatures.some((signature) => timingSafeEqual(signature, expected));
    }
    ```
  **Python, no SDK**

```python
    import hashlib
    import hmac
    import re
    import time


    def verify_b3os_signature(raw_body: bytes, header: str | None, secret: str) -> bool:
        """True only for a delivery signed with `secret` less than five minutes ago. Never raises."""
        if not isinstance(header, str) or not isinstance(raw_body, bytes):
            return False
        t = None
        signatures = []
        for part in header.split(","):
            key, sep, value = part.partition("=")
            key, value = key.strip(), value.strip()
            if not sep:
                continue
            if key == "t" and re.fullmatch(r"[0-9]{1,12}", value):
                t = int(value)
            elif key == "v1" and re.fullmatch(r"[0-9a-fA-F]{64}", value):
                signatures.append(value.lower())
        if t is None or abs(time.time() - t) > 300:
            return False
        expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
        return any(hmac.compare_digest(signature, expected) for signature in signatures)
    ```

All four reject a timestamp more than five minutes from your clock, which limits replays (the SDKs take a tolerance option). All four accept a delivery when any `v1` value matches: while you rotate a secret with `POST /v1/webhooks/secret/rotate`, deliveries carry two, one per secret, for the grace period you choose (a day by default). `POST /v1/webhooks/test` sends a signed `webhook.test` event so you can check the receiver, and `GET /v1/webhooks/deliveries` lists recent deliveries with their attempts and last status. [Webhooks](/platform/webhooks) covers both.

## Retry writes safely

Operations that change something accept an `Idempotency-Key` header. Send a unique value, such as a UUID, and retry with the same key after a timeout or a dropped connection:

```bash
curl -s https://beta-api.b3os.org/v1/sandboxes \
  -H "Authorization: Bearer $B3OS_API_KEY" \
  -H "Idempotency-Key: 6f1c2a0e-4b7d-4e55-9a51-0d8f2c8e7b21" \
  -H "Content-Type: application/json" \
  -d '{"template": "base"}'
```

- A retry with the same key, method, path and body within 24 hours returns the stored response with `Idempotent-Replayed: true`, and only to the same API key.
- The same key with a different request fails with [`idempotency_key_conflict`](/errors#idempotency_key_conflict); a retry while the first request is still running gets [`idempotency_request_in_progress`](/errors#idempotency_request_in_progress) and should wait `Retry-After`.
- Keys are scoped to your project. Responses of operations with external effects are stored durably. For creates without external effects, the stored response lives in memory and a node restart forgets it.

The API reference shows which operations take the header.