Timing and measurement

Every response tells you how long the server spent and on what. You do not need a benchmark to see it.

The Server-Timing header

Each response carries a W3C Server-Timing header with phase durations in milliseconds:

text
server-timing: edge_auth;dur=0.4, edge_upstream;dur=40.2, claim;dur=1.1, navigate;dur=31.0, extract;dur=4.2, total;dur=38.9

The values in this example are illustrative. Read it from left to right:

• edge_auth and edge_upstream are added by the edge at beta-api.b3os.org: checking your key, then the round trip to the node. Requests from inside a sandbox have no edge phases.
• The phases between them are the operation's own, such as claim, navigate and extract for a scrape that used a browser. Each operation in the API reference shows its phases in its examples.
• total is the node's time from reading the request head to writing the response head.

Timing-Allow-Origin: * is always set, so a browser exposes the header to scripts on any origin. Streaming responses (event streams, WebSockets, chunked bodies) have no total in the header, because the work is not done when the headers go out; their final phases arrive in the terminal event or on the finished job.

Bodies of operations that do work also carry the same phases as JSON, keyed <phase>_ms, with total_ms always present:

json
"timing": { "claim_ms": 1.1, "navigate_ms": 31.0, "extract_ms": 4.2, "total_ms": 36.6 }

The request log

The node keeps a record of every request it serves for 7 days, whichever way the request arrived: through the edge, from inside a sandbox, or from a host tool on the node. Look one up by the x-request-id header (or a problem's instance):

bash
curl -s "https://beta-api.b3os.org/v1/requests/$REQUEST_ID" -H "Authorization: Bearer $B3OS_API_KEY"

json
{  "id": "req_01m5t4yg394wegr3j4y01qb363",  "object": "request",  "method": "POST",  "path": "/v1/vision/scrape",  "operation_id": "visionScrape",  "status_code": 200,  "connection": "edge",  "duration_ms": 48.9,  "phases": [    { "name": "edge_auth", "source": "edge", "duration_ms": 0.4 },    { "name": "edge_upstream", "source": "edge", "duration_ms": 47.8 },    { "name": "fetch", "source": "node", "duration_ms": 41.2 },    { "name": "extract", "source": "node", "duration_ms": 3.9 },    { "name": "total", "source": "node", "duration_ms": 45.8 }  ],  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"}

This example is abridged and its values are illustrative. duration_ms runs from the first request byte to the last response byte, measured on the node; for streams, until the stream ended. connection is edge, sandbox, uds or tcp. GET /v1/requests lists recent requests and filters by product, operation, status class, key, connection and sandbox. Requests the edge answered itself, such as key management, are not in the list; fetch them by ID. See the request log.

Send a W3C traceparent header and B3OS carries it into the work it does for you, routine steps included. The record shows its trace_id.

How B3OS measures before it publishes

The numbers on this site follow fixed rules, taken from the B3OS build contract. You can hold any published figure to them.

Labels. Every figure states its provenance (measured by B3OS with raw data, or vendor, independent, competitor-run, affiliated), its vantage (same-host, same-region, or cross-region from a named place), whether it was taken on the server or the client side, the date, the sample size, and a link to the raw data. A figure B3OS cannot reproduce from raw data does not appear.

Server and client side. Server-side headline figures use first request byte to last response byte, as the request log records them. Client-side figures are labeled client side.

Load. Load generators are open loop: requests go out at a constant arrival rate, timed from when each was meant to be sent, so a slow response cannot hide the queue behind it. Every target states its arrival rate or concurrency, payload size and pool depth. Load generators run on CPU cores apart from the system under test.

Three regimes. Each target is measured back to back, after idle gaps of 100 ms and 3 seconds, and under mixed load: the other products at their budgeted load while B3IQ serves inference on the same machine. Same-node and cross-NUMA results are reported separately.

Runs. A published run holds the benchmark lock on the node and records the active services, load, versions and commit. Runs without the lock are labeled dev and never published. Each figure takes at least five runs and publishes the best run and the median of the run medians, with the sample size.

One exception, made on purpose: the dev baselines on same-host latency and benchmarks. They are dev runs, measured on the bare node before any product existed, and appear only to explain design choices, labeled as dev baselines and never as results. Their raw data sits in the B3OS beta repository and does not have a public link yet.

Comparisons. B3OS compares itself with another product only in that product's own measurement shape, and never puts a same-host figure in a row beside a client-measured competitor figure.

Benchmarks lists the targets and the state of each measurement.