Get an eval run

GET /v1/evals/runs/{eval_run_id}
One eval run with each case's result on the head and, when it ran beside it, the base. Outputs, inputs and judge explanations are not kept.

Authentication

Bearer Token (b3os_sk_ API key or b3os_pat_ token)

Path Parameters

eval_run_id string required path
Pattern ^evr_[0-9a-hjkmnp-tv-z]{26}$
Example: "evr_01m6b4k9q7d2v8x3n5c1p6t0zh"

Responses

200 The eval run.
application/json
id string REQUIRED
object string REQUIRED
Values eval_run
app string REQUIRED
suite string REQUIRED
project_id string REQUIRED
The eval project (<app>-evals) the run's sandboxes ran in.
trigger string REQUIRED
Values pull_request push
pull_request integer | null REQUIRED
commit string REQUIRED
base_commit string | null REQUIRED
base_run string | null REQUIRED
The stored run the base came from, when it was not run beside this one.
status string REQUIRED
Values running done not_run failed
reason string | null REQUIRED
Why it did not run, or stopped.
conclusion string | null REQUIRED
Values success neutral failure null
title string | null REQUIRED
head object | null REQUIRED
cases_total integer REQUIRED
Cases scored (that got an answer).
cases_passed integer REQUIRED
pass_rate number | null REQUIRED
per_scorer object REQUIRED
Each scorer's mean score.
units_per_case object REQUIRED
Mean usage per case, by unit (infer.input_tokens, judge.output_tokens).
p50_ms number | null REQUIRED
failures object REQUIRED
infra integer REQUIRED
Cases that got no answer (the service failed or timed out).
model integer REQUIRED
Cases answered and scored below the pass mark.
base object | null REQUIRED
cases_total integer REQUIRED
Cases scored (that got an answer).
cases_passed integer REQUIRED
pass_rate number | null REQUIRED
per_scorer object REQUIRED
Each scorer's mean score.
units_per_case object REQUIRED
Mean usage per case, by unit (infer.input_tokens, judge.output_tokens).
p50_ms number | null REQUIRED
failures object REQUIRED
infra integer REQUIRED
Cases that got no answer (the service failed or timed out).
model integer REQUIRED
Cases answered and scored below the pass mark.
gate object | null REQUIRED
conclusion string REQUIRED
Values success neutral failure
regressed boolean REQUIRED
change number | null REQUIRED
newly_failed integer REQUIRED
cases_ran integer REQUIRED
cases_in_file integer REQUIRED
digest string | null REQUIRED
check_run_id integer | null REQUIRED
requested_by string | null REQUIRED
created_at string (date-time) REQUIRED
finished_at string (date-time) | null REQUIRED
cases object[]
In Get an eval run only.
Array of:
id string REQUIRED
line integer REQUIRED
The case's line in the cases file.
arm string REQUIRED
Values head base
pass boolean REQUIRED
infra boolean REQUIRED
scores object REQUIRED
ms number | null REQUIRED
units object REQUIRED
error string | null REQUIRED
trace_id string | null REQUIRED
The trace the case ran under (its first failing run's, else its first). GET /v1/traces/{trace_id} shows every call the code made on that case while the request log keeps them (7 days). An id, not content.
400 The request failed validation (invalid_request, invalid_url).
401 Missing, malformed, expired or revoked credentials (unauthorized).
403 Authenticated but not allowed (forbidden, egress_denied).
404 No such eval run in your organization (eval_run_not_found).
429 A rate, concurrency or quota limit applies (rate_limited, concurrency_limited, quota_exceeded).
500 Something failed inside B3OS (internal). Retry with backoff.
503 The node or the product is not available right now (node_unavailable, product_unavailable, capacity_unavailable). 
curl -X GET 'https://beta-api.b3os.org/v1/evals/runs/evr_01m6b4k9q7d2v8x3n5c1p6t0zh' \  -H 'Authorization: Bearer YOUR_API_TOKEN'
const response = await fetch('https://beta-api.b3os.org/v1/evals/runs/evr_01m6b4k9q7d2v8x3n5c1p6t0zh', {  method: 'GET',  headers: {      "Authorization": "Bearer YOUR_API_TOKEN"  }});const data = await response.json();console.log(data);
import requestsheaders = {    'Authorization': 'Bearer YOUR_API_TOKEN'}response = requests.get('https://beta-api.b3os.org/v1/evals/runs/evr_01m6b4k9q7d2v8x3n5c1p6t0zh', headers=headers)print(response.json())
200 Response 
{  "id": "evr_01m6b4k9q7d2v8x3n5c1p6t0zh",  "object": "eval_run",  "app": "research-agent",  "suite": "answers",  "project_id": "prj_01m6b4k2zc7q9x8w3n5d1p6t0e",  "trigger": "pull_request",  "pull_request": 42,  "commit": "3f2c1b0a9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a",  "base_commit": "0a1b2c3d4e5f60718293a4b5c6d7e8f901234567",  "base_run": null,  "status": "done",  "reason": null,  "conclusion": "neutral",  "title": "1 of 2 passed (50%), no baseline yet",  "head": {    "cases_total": 2,    "cases_passed": 1,    "pass_rate": 0.5,    "per_scorer": {      "correct": 0.5    },    "units_per_case": {},    "p50_ms": 900,    "failures": {      "infra": 0,      "model": 1    }  },  "base": null,  "gate": {    "conclusion": "neutral",    "regressed": false,    "change": null,    "newly_failed": 0  },  "cases_ran": 2,  "cases_in_file": 2,  "digest": "1abebba28d77c0de",  "check_run_id": null,  "requested_by": "dana",  "created_at": "2026-11-07T10:00:00.000Z",  "finished_at": "2026-11-07T10:01:00.000Z",  "cases": [    {      "id": "paris",      "line": 1,      "arm": "head",      "pass": true,      "infra": false,      "scores": {        "correct": 1      },      "ms": 880,      "units": {},      "error": null,      "trace_id": "0af7651916cd43dd8448eb211c80319c"    },    {      "id": "rome",      "line": 2,      "arm": "head",      "pass": false,      "infra": false,      "scores": {        "correct": 0      },      "ms": 920,      "units": {},      "error": null,      "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"    }  ]}
Previous List eval runs
Next List alerts
Ask a question... ⌘I