# Manage runs

> list, status, cancel, and download against SELFBENCH_API_URL, including phase polling, create-only writes, and integrity headers.

- Repository: mupt-ai/self-bench
- GitHub: https://github.com/mupt-ai/self-bench
- Human docs: https://grok-wiki.com/public/docs/mupt-ai-self-bench-62fff2be9bcc
- Complete Markdown: https://grok-wiki.com/public/docs/mupt-ai-self-bench-62fff2be9bcc/llms-full.txt

## Source Files

- `src/cli.ts`
- `src/api.ts`
- `src/run-wait.ts`
- `src/contracts.ts`
- `tests/run-wait.test.ts`
- `docs/operations.md`

---

---
title: "Manage runs"
description: "list, status, cancel, and download against SELFBENCH_API_URL, including phase polling, create-only writes, and integrity headers."
---

`self-bench list`, `status`, `cancel`, and `download` are one-shot HTTP clients of the API at `SELFBENCH_API_URL` (default `http://127.0.0.1:8080`). They send `Authorization: Bearer $SELFBENCH_API_TOKEN` when that variable is set, print the JSON body, and never own Temporal execution. Closing the CLI — including a waiting `self-bench run --wait` or `--output` process — does not cancel the workflow.

<Note>
The local Compose API always has a token (`SELFBENCH_API_TOKEN` or the compose default `selfbench-local`). A CLI that omits the token receives `401` `{ "error": "unauthorized" }`.
</Note>

## Prerequisites

<Steps>
<Step title="Point the CLI at a running API">
```bash
export SELFBENCH_API_URL=http://127.0.0.1:8080
export SELFBENCH_API_TOKEN="$SELFBENCH_API_TOKEN"
```

`SELFBENCH_API_URL` is read only by the CLI. The API process binds `SELFBENCH_API_HOST`:`SELFBENCH_API_PORT` (compose publishes `127.0.0.1:8080`).
</Step>
<Step title="Confirm the API is up">
```bash
curl -sS "$SELFBENCH_API_URL/healthz"
```

Expected: `{ "ok": true }`. `/healthz` is unauthenticated. Every `/v1/runs` route requires the Bearer token when the API was started with `SELFBENCH_API_TOKEN`.
</Step>
</Steps>

<ParamField body="SELFBENCH_API_URL" type="string">
CLI base URL. Default `http://127.0.0.1:8080`.
</ParamField>

<ParamField body="SELFBENCH_API_TOKEN" type="string">
Shared secret. When set, the CLI adds `Authorization: Bearer …`. The API compares it with a timing-safe equality check. A non-loopback API bind refuses to start without this token.
</ParamField>

## Commands

| Command | HTTP | CLI output |
| --- | --- | --- |
| `self-bench list` | `GET /v1/runs` | Temporal execution summaries (JSON array) |
| `self-bench status RUN_ID` | `GET /v1/runs/:runId` | `RunStatus` JSON |
| `self-bench cancel RUN_ID` | `POST /v1/runs/:runId/cancel` | `{ runId, cancellationRequested: true }` |
| `self-bench download RUN_ID OUTPUT.tar.gz` | `GET /v1/runs/:runId/export` | `{ runId, output }` after a verified write |

```bash
self-bench list
self-bench status RUN_ID
self-bench cancel RUN_ID
self-bench download RUN_ID ./self-bench-evals.tar.gz
```

`RUN_ID` is the first positional argument. `download` requires a second path. These commands take no flags.

`self-bench run --wait` and `self-bench run --output PATH` are the only surfaces that poll `GET /v1/runs/:runId`. `--output` implies `--wait`, then calls the same download path.

## Run IDs

API run paths accept only `^[a-z0-9][a-z0-9-]{2,62}$` (3–63 characters, start with a letter or digit). A custom `--run-id` on submit must match that regex. Generated IDs look like `sb-20260814153022-a1b2c3d4`.

An ID that fails the path regex never reaches Temporal: the API returns `404` `{ "error": "not found" }`. Generation workflows that fail must be resubmitted under a **new** run ID. There is no remote delete route.

## List runs

`GET /v1/runs` walks Temporal visibility for `WorkflowType = 'selfBenchRunWorkflow'` and returns every matching execution. There is no pagination, filter, or limit.

<ResponseExample>
```json
[
  {
    "runId": "sb-20260814153022-a1b2c3d4",
    "status": "RUNNING",
    "startedAt": "2026-08-14T15:30:22.000Z",
    "closedAt": null
  }
]
```
</ResponseExample>

<ResponseField name="runId" type="string">
Temporal workflow ID. Same string as the submitted run ID.
</ResponseField>

<ResponseField name="status" type="string">
Temporal execution name (`RUNNING`, `COMPLETED`, `FAILED`, `CANCELED`, `TERMINATED`, `TIMED_OUT`, …). This is **not** the application `phase`.
</ResponseField>

<ResponseField name="startedAt" type="string">
ISO-8601 start time.
</ResponseField>

<ResponseField name="closedAt" type="string">
ISO-8601 close time when the execution has closed; omitted or null while running.
</ResponseField>

<Warning>
`list.status` and `status.phase` are different vocabularies. Temporal `CANCELED` maps to application phase `cancelled`. Use `self-bench status` for accepted/rejected counts, per-candidate stages, and export readiness.
</Warning>

## Read status

`GET /v1/runs/:runId` returns the workflow `status` query when the execution is `RUNNING` or the query already reports a terminal application phase. Otherwise the API overlays the Temporal execution onto `phase`.

If the query handler is unavailable but `describe()` succeeds, the body shrinks to `{ runId, phase }`. `RUNNING` in that fallback is reported as `queued`.

<ResponseExample>
```json
{
  "runId": "sb-20260814153022-a1b2c3d4",
  "phase": "authoring",
  "requested": 20,
  "requestedByDifficulty": { "easy": 5, "medium": 10, "hard": 5 },
  "discovered": 24,
  "accepted": 3,
  "rejected": 2,
  "tasks": [
    {
      "taskId": "project-pr-123",
      "candidateId": "project-pr-123",
      "difficulty": "medium",
      "status": "validating"
    }
  ],
  "discovery": {
    "wave": 0,
    "totalShards": 8,
    "completedShards": 8,
    "failedShards": 0,
    "candidates": 24
  }
}
```
</ResponseExample>

| Field | Meaning |
| --- | --- |
| `phase` | Application phase: `queued`, `discovering`, `authoring`, `exporting`, `complete`, `blocked`, `failed`, `cancelled` |
| `requested` | Sum of the three tier budgets |
| `requestedByDifficulty` | `{ easy, medium, hard }` budgets |
| `discovered` | Unique candidates collected so far |
| `accepted` / `rejected` | Counts derived from `tasks[].status` |
| `tasks[]` | Per-candidate `authoring` → `auditing` → `validating` → `reviewing` → `repairing` → `accepted` / `rejected` / `infrastructure_failed` |
| `discovery` | Current wave, eight shards, completed/failed shard counts, candidate count |
| `export` | Present after a successful export: `{ uri, sha256, sizeBytes, contentType }` |
| `error` | Set on `blocked` / `failed` (and on some Temporal overlays such as `Temporal workflow timed_out`) |

A completed status includes `export` with `contentType` `application/gzip` and a 64-character lowercase hex `sha256`. That object is what `download` and the review UI consume.

Phase meanings, shard refill, and terminal `blocked` / `failed` / `cancelled` behavior live on [Run lifecycle](/run-lifecycle).

## Phase polling

`waitForRun` polls `GET /v1/runs/:runId` every **2000 ms**. It emits `onPhase` only when `phase` changes. `self-bench run` writes those events to **stderr** as one JSON object per change:

```json
{"runId":"sb-20260814153022-a1b2c3d4","phase":"authoring","accepted":1,"rejected":0}
```

```mermaid
stateDiagram-v2
  [*] --> Poll
  Poll --> EmitPhase: phase changed
  EmitPhase --> Decide
  Poll --> Decide: same phase
  Decide --> Complete: phase == complete
  Decide --> Fail: blocked or failed or cancelled
  Decide --> Sleep: otherwise
  Sleep --> Poll: 2000 ms
```

| Terminal `phase` | Waiter behavior |
| --- | --- |
| `complete` | Returns the status. `--output` then downloads. `--wait` without `--output` prints the status on stdout. |
| `blocked`, `failed`, `cancelled` | Throws `SelfBench run <phase>` plus `: <error>` when `error` is a string. No download. |
| Missing `phase` | Throws `SelfBench status response is missing its phase`. |

`self-bench status` does not poll. Re-run it, or use `run --wait` / `--output`.

## Cancel a run

:::endpoint POST /v1/runs/:runId/cancel Request Temporal cancellation
**Auth:** Bearer token when configured.

**Success:** `202`

```json
{
  "runId": "sb-20260814153022-a1b2c3d4",
  "cancellationRequested": true
}
```

The handler calls Temporal `workflow.cancel()` and returns immediately. In-flight activities use `WAIT_CANCELLATION_COMPLETED`. The workflow sets `phase` to `cancelled` when it observes cancellation, then rethrows so Temporal records `CANCELED`.

A stopped worker does not finish the cancel until it restarts. A waiting CLI is not cancelled by Ctrl-C; call this route (or `self-bench cancel`) explicitly.
:::

`self-bench list` still shows Temporal `CANCELED` after the execution closes. Failed generation is not retryable under the same ID.

## Download an export

:::endpoint GET /v1/runs/:runId/export Stream the completed archive
**Auth:** Bearer token when configured.

**Success headers:**

| Header | Value |
| --- | --- |
| `content-type` | Artifact `contentType` (`application/gzip`) |
| `content-length` | Artifact `sizeBytes` |
| `content-disposition` | `attachment; filename="selfbench-<runId>.tar.gz"` |
| `x-content-sha256` | Artifact SHA-256 (64 hex chars) |

**Errors:** `409` `{ "error": "run export is not ready" }` when `status.export` is missing. `401` if the token is required and missing or wrong.

The API streams `artifacts.openRead(export)`, which checks stored length and SHA-256 while reading. A store integrity failure becomes a mid-stream error after headers may already be sent.
:::

<CodeGroup>
```bash CLI
self-bench download sb-20260814153022-a1b2c3d4 ./self-bench-evals.tar.gz
```

```bash curl
curl -fL \
  -H "Authorization: Bearer $SELFBENCH_API_TOKEN" \
  -D - \
  -o ./self-bench-evals.tar.gz \
  "$SELFBENCH_API_URL/v1/runs/sb-20260814153022-a1b2c3d4/export"
```
</CodeGroup>

Successful CLI stdout:

```json
{
  "runId": "sb-20260814153022-a1b2c3d4",
  "output": "/absolute/path/to/self-bench-evals.tar.gz"
}
```

The hosted review UI lists the same `/v1/runs` array and fetches `/export` for a completed run. Cleaning or dropping tasks is a review-UI concern, not a CLI download flag.

## Create-only writes and integrity

```mermaid
sequenceDiagram
  participant CLI as self-bench download
  participant API as HTTP API
  participant WF as Temporal status query
  participant Store as Artifact store
  CLI->>API: GET /v1/runs/:runId/export
  API->>WF: query status
  alt export missing
    API-->>CLI: 409 run export is not ready
  else export present
    API->>Store: openRead(export)
    API-->>CLI: 200 + x-content-sha256 + body
    CLI->>CLI: open(output, wx)
    CLI->>CLI: stream SHA-256
    alt header missing or hash mismatch or write error
      CLI->>CLI: rm output
      CLI-->>CLI: throw
    else digest matches header
      CLI-->>CLI: print runId and output
    end
  end
```

Rules the CLI enforces:

1. **Integrity header required.** Missing `x-content-sha256` or an empty body throws `SelfBench API returned an export without integrity metadata`.
2. **Create-only file.** The destination is opened with `wx`. An existing path fails (typically `EEXIST`); the CLI does not overwrite.
3. **Streamed SHA-256.** Bytes are hashed while written. The hex digest must equal `x-content-sha256`.
4. **Failed writes are deleted.** Any error after `open` — including `downloaded export failed its SHA-256 integrity check` — `rm`s the destination so a partial archive cannot be mistaken for a good export.

Artifact `put` / `putFile` on the server also use create-only (`wx` / exclusive link). A key may be rewritten only when the existing bytes already match.

Exports contain source snapshots, held-out tests, and reference solutions. They are unencrypted. Treat the output path as sensitive.

## HTTP mapping

| Method | Path | Status | Body |
| --- | --- | --- | --- |
| `GET` | `/healthz` | 200 | `{ ok: true }` (no auth) |
| `GET` | `/v1/runs` | 200 | Execution array |
| `GET` | `/v1/runs/:runId` | 200 | `RunStatus` |
| `POST` | `/v1/runs/:runId/cancel` | 202 | `{ runId, cancellationRequested }` |
| `GET` | `/v1/runs/:runId/export` | 200 / 409 | gzip stream or `{ error }` |
| `POST` | `/v1/runs` | 202 | `{ runId }` — submit only; see [Build a benchmark](/build-a-benchmark) |

JSON error envelope: `{ "error": "<message>" }` with `400` (Zod / `SyntaxError`), `401`, `404`, `409`, or `500`. The CLI throws `error` when present, otherwise `SelfBench API returned <status>`.

A Temporal describe/query failure for an unknown ID surfaces as `500` with the Temporal message. Duplicate `POST /v1/runs` for an existing workflow ID also fails at Temporal start (treat as a new-ID problem, not an in-place retry).

Workflows are started with `workflowExecutionTimeout: "14 days"`. A timeout overlays `phase: "failed"` and, if the query has no `error`, `Temporal workflow timed_out`.

## Errors

| Symptom | Cause | What to do |
| --- | --- | --- |
| `unauthorized` | Token configured on the API, missing or wrong on the CLI | Export the same `SELFBENCH_API_TOKEN` the API process has. Compose default is `selfbench-local`. |
| `run ID is required` / `output path is required` | Missing positional args | `status`/`cancel` need `RUN_ID`; `download` needs `RUN_ID` and a path. |
| `not found` | Path failed the run-ID regex, or no such route | Check the ID charset and length. |
| `run export is not ready` | `phase` is not `complete`, or `export` is absent | Wait, or inspect `self-bench status`. Do not retry download on `blocked` / `failed` / `cancelled`. |
| `downloaded export failed its SHA-256 integrity check` | Stream digest ≠ `x-content-sha256` | The partial file is removed. Retry once; persistent mismatch is a store/transport problem. |
| `SelfBench API returned an export without integrity metadata` | Response lacked `x-content-sha256` or a body | The API always sets the header on a successful export; a proxy or mock is dropping it. |
| `EEXIST` | Destination already exists | Choose a new path. Download will not overwrite. |
| `SelfBench run blocked: candidate pool exhausted; missing easy=…` | Discovery could not fill the requested budgets | New run ID; see [Run lifecycle](/run-lifecycle) and [Troubleshooting](/troubleshooting). |
| `SelfBench run failed: …` / `SelfBench run cancelled` | Waiter saw a terminal unsuccessful phase | Read `status.error`. Cancelled waits are expected after `self-bench cancel`. |
| Work appears stuck after `list` shows `RUNNING` | Worker or Compose stack stopped | Restart the worker. History and artifacts stay on the named volumes. |
| Need to drop a completed run | No `DELETE` route | Delete the local artifact volume or the GCS prefix with operator tooling. |

## Related pages

<CardGroup>
<Card title="Run lifecycle" href="/run-lifecycle">
Phases, shard discovery, budgets versus accepted tasks, and terminal blocked/failed/cancelled states.
</Card>
<Card title="Build a benchmark" href="/build-a-benchmark">
`self-bench run` submission, `--run-id`, `--wait`, and `--output`.
</Card>
<Card title="HTTP API" href="/http-api">
Full route table, Bearer auth, provenance limits, and export headers.
</Card>
<Card title="Review exports" href="/review-exports">
API-hosted review UI, drop tasks, and a cleaned tar.gz.
</Card>
<Card title="Export format" href="/export-format">
Outer archive layout, `manifest.json`, and SHA-256 checks.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
409 export-not-ready, SHA-256 mismatch, and leftover Docker resources.
</Card>
</CardGroup>