List eval runs

GET /v1/evals/runs
The eval runs of your organization's applications, newest first, without their cases.

Authentication

Bearer Token (b3os_sk_ API key or b3os_pat_ token)

Query Parameters

app string optional query
An application's name, the name in its b3os.yaml.
Min length 1Max length 63Pattern ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$
suite string optional query
An application's name, the name in its b3os.yaml.
Min length 1Max length 63Pattern ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$
pull_request integer optional query
Min 1
commit string optional query
Pattern ^[0-9a-f]{40}$
limit integer optional query
Page size, from 1 to 100.
Default 20Min 1Max 100
cursor string optional query
Opaque cursor from the previous page's next_cursor. Omit it for the first page.
Max length 512

Responses

200 A page of eval runs.
application/json
object string REQUIRED
Values list
data object[] REQUIRED
Array of:
id string REQUIRED
object string REQUIRED
Values eval_run
app string REQUIRED
suite string REQUIRED
project_id string REQUIRED
The eval project (<app>-evals) the run's sandboxes ran in.
trigger string REQUIRED
Values pull_request push
pull_request integer | null REQUIRED
commit string REQUIRED
base_commit string | null REQUIRED
base_run string | null REQUIRED
The stored run the base came from, when it was not run beside this one.
status string REQUIRED
Values running done not_run failed
reason string | null REQUIRED
Why it did not run, or stopped.
conclusion string | null REQUIRED
Values success neutral failure null
title string | null REQUIRED
head object | null REQUIRED
cases_total integer REQUIRED
Cases scored (that got an answer).
cases_passed integer REQUIRED
pass_rate number | null REQUIRED
per_scorer object REQUIRED
Each scorer's mean score.
units_per_case object REQUIRED
Mean usage per case, by unit (infer.input_tokens, judge.output_tokens).
p50_ms number | null REQUIRED
failures object REQUIRED
infra integer REQUIRED
Cases that got no answer (the service failed or timed out).
model integer REQUIRED
Cases answered and scored below the pass mark.
base object | null REQUIRED
cases_total integer REQUIRED
Cases scored (that got an answer).
cases_passed integer REQUIRED
pass_rate number | null REQUIRED
per_scorer object REQUIRED
Each scorer's mean score.
units_per_case object REQUIRED
Mean usage per case, by unit (infer.input_tokens, judge.output_tokens).
p50_ms number | null REQUIRED
failures object REQUIRED
infra integer REQUIRED
Cases that got no answer (the service failed or timed out).
model integer REQUIRED
Cases answered and scored below the pass mark.
gate object | null REQUIRED
conclusion string REQUIRED
Values success neutral failure
regressed boolean REQUIRED
change number | null REQUIRED
newly_failed integer REQUIRED
cases_ran integer REQUIRED
cases_in_file integer REQUIRED
digest string | null REQUIRED
check_run_id integer | null REQUIRED
requested_by string | null REQUIRED
created_at string (date-time) REQUIRED
finished_at string (date-time) | null REQUIRED
cases object[]
In Get an eval run only.
Array of:
id string REQUIRED
line integer REQUIRED
The case's line in the cases file.
arm string REQUIRED
Values head base
pass boolean REQUIRED
infra boolean REQUIRED
scores object REQUIRED
ms number | null REQUIRED
units object REQUIRED
error string | null REQUIRED
trace_id string | null REQUIRED
The trace the case ran under (its first failing run's, else its first). GET /v1/traces/{trace_id} shows every call the code made on that case while the request log keeps them (7 days). An id, not content.
next_cursor string | null REQUIRED
400 The request failed validation (invalid_request, invalid_url).
401 Missing, malformed, expired or revoked credentials (unauthorized).
403 Authenticated but not allowed (forbidden, egress_denied).
429 A rate, concurrency or quota limit applies (rate_limited, concurrency_limited, quota_exceeded).
500 Something failed inside B3OS (internal). Retry with backoff.
503 The node or the product is not available right now (node_unavailable, product_unavailable, capacity_unavailable). 
curl -X GET 'https://beta-api.b3os.org/v1/evals/runs' \  -H 'Authorization: Bearer YOUR_API_TOKEN'
const response = await fetch('https://beta-api.b3os.org/v1/evals/runs', {  method: 'GET',  headers: {      "Authorization": "Bearer YOUR_API_TOKEN"  }});const data = await response.json();console.log(data);
import requestsheaders = {    'Authorization': 'Bearer YOUR_API_TOKEN'}response = requests.get('https://beta-api.b3os.org/v1/evals/runs', headers=headers)print(response.json())
200 Response 
{  "object": "list",  "data": [    {      "id": "evr_01m6b4k9q7d2v8x3n5c1p6t0zh",      "object": "eval_run",      "app": "research-agent",      "suite": "answers",      "project_id": "prj_01m6b4k2zc7q9x8w3n5d1p6t0e",      "trigger": "pull_request",      "pull_request": 42,      "commit": "3f2c1b0a9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a",      "base_commit": "0a1b2c3d4e5f60718293a4b5c6d7e8f901234567",      "base_run": "evr_01m6b3z1q7d2v8x3n5c1p6t0aa",      "status": "done",      "reason": null,      "conclusion": "success",      "title": "18 of 20 passed (90%), base 19 of 20 (95%); cost per case +4%",      "head": {        "cases_total": 20,        "cases_passed": 18,        "pass_rate": 0.9,        "per_scorer": {          "correct": 0.9        },        "units_per_case": {          "infer.input_tokens": 1830        },        "p50_ms": 1210,        "failures": {          "infra": 0,          "model": 2        }      },      "base": {        "cases_total": 20,        "cases_passed": 19,        "pass_rate": 0.95,        "per_scorer": {          "correct": 0.95        },        "units_per_case": {          "infer.input_tokens": 1760        },        "p50_ms": 1180,        "failures": {          "infra": 0,          "model": 1        }      },      "gate": {        "conclusion": "success",        "regressed": false,        "change": -0.05,        "newly_failed": 1      },      "cases_ran": 20,      "cases_in_file": 64,      "digest": "1abebba28d77c0de",      "check_run_id": 2100000003,      "requested_by": "dana",      "created_at": "2026-11-07T10:00:00.000Z",      "finished_at": "2026-11-07T10:03:10.000Z"    }  ],  "next_cursor": null}
Previous Get a trace
Next Get an eval run
Ask a question... ⌘I