# Start the local stack

> self-bench up and down: Compose services, --backend docker or modal, --modal-config, named volumes, and what happens when the worker stops.

- Repository: mupt-ai/self-bench
- GitHub: https://github.com/mupt-ai/self-bench
- Human docs: https://grok-wiki.com/public/docs/mupt-ai-self-bench-62fff2be9bcc
- Complete Markdown: https://grok-wiki.com/public/docs/mupt-ai-self-bench-62fff2be9bcc/llms-full.txt

## Source Files

- `src/cli.ts`
- `compose.yaml`
- `Dockerfile`
- `Dockerfile.sandbox`
- `src/temporal/worker-main.ts`
- `tests/cli.test.ts`

---

---
title: "Start the local stack"
description: "self-bench up and down: Compose services, --backend docker or modal, --modal-config, named volumes, and what happens when the worker stops."
---

`self-bench up` starts the packaged Compose project `selfbench`. It resolves `compose.yaml` next to the installed CLI (parent of `dist/cli.js`), optionally builds `selfbench-sandbox:local`, then runs `docker compose --file <compose.yaml> up -d --build`. `self-bench down` runs `docker compose down` against the same file and does **not** pass `--volumes`, so Temporal history and generated artifacts stay on the named volumes.

The command sets both `SELFBENCH_EXECUTION_BACKEND` and `SELFBENCH_HARBOR_ENVIRONMENT` to the same `--backend` value. Sandbox execution and Harbor `nop` / `oracle` gates therefore use one backend.

<Warning>
Export `SELFBENCH_API_TOKEN` **before** `self-bench up`. Compose binds the API at `0.0.0.0:8080` inside the container and defaults the token to `selfbench-local` when the host env is unset. The CLI sends `Authorization: Bearer` only when `SELFBENCH_API_TOKEN` is set in the calling shell. A running API with a token and a CLI without one returns `401 unauthorized`.
</Warning>

## Prerequisites

- Docker Engine with the Compose plugin
- Node.js 22+ (or a global `self-bench` install that already ships `dist/cli.js`)
- For `--backend modal`: a Modal profile at `~/.modal.toml`, or `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET`
- Worker credentials in the host environment (Compose interpolates them into the worker only): `OPENAI_API_KEY`, `GH_TOKEN`

Install and auth steps live on [Installation](/installation). This page assumes those are already set.

```bash
export OPENAI_API_KEY=...
export GH_TOKEN="$(gh auth token)"
export SELFBENCH_API_TOKEN="$(openssl rand -hex 24)"
```

## Commands

```text
self-bench up [--backend docker|modal] [--modal-config PATH]
self-bench down
```

<ParamField body="--backend" type="docker | modal" default="docker">
Selects sandbox execution **and** Harbor `--env`. Any other value fails with `--backend must be "docker" or "modal"`.
</ParamField>

<ParamField body="--modal-config" type="path">
Host path mounted read-only at `/root/.modal.toml` in the worker. Allowed only with `--backend modal`. Default when omitted: `~/.modal.toml` (`os.homedir()` + `/.modal.toml`).
</ParamField>

`down` takes no flags.

<RequestExample>
```bash
self-bench up --backend modal
```
</RequestExample>

<ResponseExample>
```text
SelfBench is running with the modal backend at http://127.0.0.1:8080
```
</ResponseExample>

On success, point later CLI commands at that listener (this is also the CLI default):

```bash
export SELFBENCH_API_URL=http://127.0.0.1:8080
```

## Start the stack

<Tabs>
<Tab title="Modal (recommended)">
<Steps>
<Step title="Authenticate Modal">
```bash
modal token new
```
</Step>
<Step title="Bring the stack up">
```bash
self-bench up --backend modal
```

Does **not** build `Dockerfile.sandbox`. Sets `SELFBENCH_MODAL_CONFIG_PATH` to `~/.modal.toml` unless `--modal-config` overrides it.
</Step>
<Step title="Override the profile path if needed">
```bash
self-bench up --backend modal --modal-config /absolute/path/to/.modal.toml
```
</Step>
<Step title="Verify the API">
```bash
curl -s http://127.0.0.1:8080/healthz
```

Expected body:

```json
{
  "ok": true
}
```
</Step>
</Steps>
</Tab>
<Tab title="Docker sandboxes">
<Steps>
<Step title="Bring the stack up">
```bash
self-bench up --backend docker
```

Builds `Dockerfile.sandbox` as `selfbench-sandbox:local` **before** Compose. Compose then sets `SELFBENCH_DOCKER_IMAGE=selfbench-sandbox:local`.
</Step>
<Step title="Confirm the worker can talk to the host daemon">
The worker container starts as `root`, mounts `/var/run/docker.sock`, and on boot runs `docker info` and `docker compose version` (30s timeout each) because execution or Harbor is `docker`. Failure exits the process; Compose `restart: on-failure` retries it.
</Step>
<Step title="Verify the API">
```bash
curl -s http://127.0.0.1:8080/healthz
```
</Step>
</Steps>
</Tab>
</Tabs>

`up` also injects `SELFBENCH_BUILD_COMMIT`: the current `SELFBENCH_BUILD_COMMIT` env if set, else the baked `buildCommit` from package metadata, else `git rev-parse HEAD` in the package root. A missing or all-zero SHA fails with `SelfBench was built without commit metadata; set SELFBENCH_BUILD_COMMIT to a full commit SHA`.

Every `up` rebuilds `selfbench:local` from the packaged `Dockerfile` (`--build`).

## Compose project

The packaged file is `compose.yaml` with top-level `name: selfbench`. That name is stable across global npm/bun install paths, so the same volumes are reused.

:::files
self-bench package (parent of dist/cli.js)
├── compose.yaml
├── Dockerfile              → image selfbench:local (API + worker)
├── Dockerfile.sandbox      → image selfbench-sandbox:local (docker backend only)
└── dist/
    ├── api-main.js         → API default CMD
    └── temporal/worker-main.js
:::

| Service | Image | Command | Host publish | Restart |
| --- | --- | --- | --- | --- |
| `postgres` | `postgres:17.6-bookworm` | image default | none | none |
| `temporal` | `temporalio/auto-setup:1.29.6` | image default | `127.0.0.1:7233` | `on-failure` |
| `api` | `selfbench:local` | `node dist/api-main.js` | `127.0.0.1:8080` | `on-failure` |
| `worker` | `selfbench:local` | `node dist/temporal/worker-main.js` | none | `on-failure` |

Postgres becomes healthy via `pg_isready -U temporal` (5s interval, 20 retries). Temporal waits for that. API and worker only `depends_on: temporal` with no health gate: if Temporal is not accepting connections yet, the Node process exits and Compose restarts it.

There is no Temporal Web UI in this project. Postgres is not published on the host. Local DB user and password are both `temporal` (`POSTGRES_DB=temporal`).

```mermaid
flowchart TB
  subgraph host [Host]
    CLI["self-bench CLI"]
    Sock["/var/run/docker.sock"]
    ModalFile["SELFBENCH_MODAL_CONFIG_PATH or ~/.modal.toml"]
    Creds["OPENAI_API_KEY GH_TOKEN MODAL_TOKEN_*"]
  end

  subgraph compose [Compose project selfbench]
    PG["postgres"]
    Temporal["temporal :7233"]
    API["api :8080"]
    Worker["worker"]
  end

  subgraph volumes [Named volumes]
    PGVol["selfbench_temporal-postgres"]
    ArtVol["selfbench_artifacts"]
  end

  subgraph exec [Execution]
    Sandbox["selfbench-sandbox:local"]
    ModalSB["Modal sandboxes"]
  end

  CLI -->|"compose up -d --build"| compose
  CLI -->|"docker build -f Dockerfile.sandbox"| Sandbox
  PG --> PGVol
  Temporal --> PG
  API --> Temporal
  API --> ArtVol
  Worker --> Temporal
  Worker --> ArtVol
  Worker --> Sock
  Worker --> ModalFile
  Creds --> Worker
  Worker --> Sandbox
  Worker --> ModalSB
```

## What each service gets

Compose shares an environment anchor (`selfbench-environment`) between API and worker, then adds credentials only on the worker.

| Key | API | Worker | Source |
| --- | --- | --- | --- |
| `SELFBENCH_API_HOST` | `0.0.0.0` | same | Compose |
| `SELFBENCH_API_TOKEN` | `${SELFBENCH_API_TOKEN:-selfbench-local}` | same | Host env |
| `SELFBENCH_TEMPORAL_ADDRESS` | `temporal:7233` | same | Compose |
| `SELFBENCH_ARTIFACT_BACKEND` | `local` | same | Compose |
| `SELFBENCH_ARTIFACT_DIR` | `/var/lib/selfbench/artifacts` | same | Compose |
| `SELFBENCH_EXECUTION_BACKEND` | from `--backend` | same | `self-bench up` |
| `SELFBENCH_HARBOR_ENVIRONMENT` | from `--backend` | same | `self-bench up` |
| `SELFBENCH_DOCKER_IMAGE` | `selfbench-sandbox:local` | same | Compose |
| `SELFBENCH_BUILD_COMMIT` | from CLI | same | `self-bench up` |
| `GH_TOKEN` | — | `${GH_TOKEN:-}` | Host env |
| `OPENAI_API_KEY` | — | `${OPENAI_API_KEY:-}` | Host env |
| `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` | — | interpolated | Host env |
| `SELFBENCH_PI_AUTH_JSON` / `SELFBENCH_CODEX_AUTH_JSON` | — | interpolated | Host env |

The API image default user is `node`. The worker service overrides `user: root` and `group_add: ${DOCKER_GID:-0}` so it can use the mounted Docker socket. The API never receives the socket or model / GitHub / Modal credentials.

Inside the container the worker polls Temporal namespace `default` and task queue `selfbench-dev` (config defaults; Compose does not override them). Startup log:

```text
SelfBench worker polling default/selfbench-dev with activity concurrency N
```

`N` is `1` for docker and `20` for modal unless `SELFBENCH_ACTIVITY_CONCURRENCY` is set (1–100).

Unauthenticated surfaces on the published API:

| Method | Path | Behavior |
| --- | --- | --- |
| `GET` | `/healthz` | `{ "ok": true }` |
| `GET` | `/` and `/assets/*` | review UI static files |

Every `/v1/*` route requires `Authorization: Bearer $SELFBENCH_API_TOKEN` when the token is configured (always true in this Compose file).

## Backends

`self-bench up` writes both backend env vars to the **same** value. Harbor validation calls `harbor run --env <backend>`.

| `--backend` | Sandbox image | Harbor `--env` | Activity concurrency | Extra `up` work | Worker Docker probe |
| --- | --- | --- | --- | --- | --- |
| `docker` | `selfbench-sandbox:local` | `docker` | 1 | `docker build -f Dockerfile.sandbox -t selfbench-sandbox:local` | `docker info` + `docker compose version` |
| `modal` | Modal image default `node:22-bookworm` | `modal` | 20 | mount Modal profile | skipped |

`--modal-config` with `--backend docker` fails: `--modal-config requires --backend modal`.

When `--backend` is not `modal`, Compose leaves `SELFBENCH_MODAL_CONFIG_PATH` unset and the worker mount becomes `/dev/null:/root/.modal.toml:ro`.

<Note>
Empty `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` (unset or whitespace) are deleted at worker process start so they cannot override a valid mounted `~/.modal.toml`. Non-empty token env vars win over the profile.
</Note>

Resource limits, inactivity timeouts, and Modal app/image knobs are on [Execution backends](/execution-backends).

## Named volumes

| Compose volume | Docker volume name | Mount | Holds |
| --- | --- | --- | --- |
| `temporal-postgres` | `selfbench_temporal-postgres` | `postgres:/var/lib/postgresql/data` | Temporal workflow history |
| `artifacts` | `selfbench_artifacts` | API and worker `/var/lib/selfbench/artifacts` | Provenance, per-run artifacts, export tarballs |

`self-bench down` is `docker compose --file <compose.yaml> down` with no `--volumes`. Containers and the default network go away; both volumes remain.

There are no application database migrations. Back up those two volumes before replacing images if history or exports must survive an upgrade.

Self-bench has no HTTP delete route. Wipe data with ordinary Docker volume tooling, for example `docker volume rm selfbench_artifacts selfbench_temporal-postgres` after `down`.

## What happens when the worker stops

Closing a waiting `self-bench run --wait` / `--output` process does **not** cancel the Temporal workflow. The API already started it with `workflowId = runId` and `workflowExecutionTimeout: "14 days"`.

```mermaid
stateDiagram-v2
  [*] --> ContainersDown
  ContainersDown --> WorkerPolling: self-bench up
  WorkerPolling --> WorkflowPaused: worker exits or stack stops
  WorkflowPaused --> WorkerPolling: worker process returns
  WorkerPolling --> ContainersDown: self-bench down
```

| Event | Workflows | In-flight activities | Volumes | Next action |
| --- | --- | --- | --- | --- |
| Worker process crash | Persist in Temporal / Postgres | Heartbeats stop | Unchanged | Compose `restart: on-failure` starts the worker again |
| `docker stop` on worker (not a non-zero exit) | Persist | Heartbeats stop | Unchanged | Stays down until `docker compose start` or another `self-bench up` |
| Whole stack crash / `self-bench down` | Persist on `selfbench_temporal-postgres` | Activities abort | Persist | `self-bench up` remounts volumes; worker resumes `default/selfbench-dev` |
| Host crash during a Docker sandbox | Persist | Sandbox may be left on the host | Persist | Inspect leftover `selfbench-<runId>-<stage>-<suffix>` containers/volumes |

Activity proxy timeouts (so a dead worker does not hang a run forever):

| Activities | `startToCloseTimeout` | `heartbeatTimeout` | Retry |
| --- | --- | --- | --- |
| Author, validate, review, repair, audit, export | 7 hours | 10 minutes | 4 attempts, 5s → 2m backoff |
| Discovery shards | 1 hour | 10 minutes | 3 attempts, 5s → 1m backoff |
| Validation-repair | 2 hours | 10 minutes | 1 attempt |

After ~10 minutes without a heartbeat, Temporal marks the activity failed and retries according to the table. The worker does not resume the interrupted OS process; it takes the next retry.

Docker sandboxes are created as disposable containers and volumes named `selfbench-<runId>-<stage>-<suffix>` and are `docker rm` / `docker volume rm` on normal completion. A crash can leave those host resources for an operator to remove. Modal sandboxes are disposable on the Modal side.

`postgres` has **no** `restart:` policy. If that container dies and stays down, Temporal and then the API/worker fail until Postgres is running again.

## Stop the stack

```bash
self-bench down
```

Use `self-bench list` / `self-bench status RUN_ID` after the next `up` to see runs that were in flight. Cancel only with `self-bench cancel RUN_ID`.

## Linux Docker group

If the worker cannot use the mounted socket, set `DOCKER_GID` to the host `docker` group id before `up`:

```bash
export DOCKER_GID="$(getent group docker | cut -d: -f3)"
self-bench up --backend docker
```

Compose default is `0`.

## Failure modes

<AccordionGroup>
<Accordion title="401 unauthorized from list / run / status">
The API container has a token (`selfbench-local` or whatever was exported at `up` time). Export the **same** `SELFBENCH_API_TOKEN` in the CLI shell. Changing the host token after `up` does not rotate the running API; run `self-bench down` and `up` again.
</Accordion>
<Accordion title="Worker crash loop on docker info">
`--backend docker` (or a compose override that leaves Harbor on docker) requires a working host Docker API from inside the worker. Check `/var/run/docker.sock` and `DOCKER_GID`.
</Accordion>
<Accordion title="Modal profile ignored">
Blank `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` are stripped. Non-empty values override `/root/.modal.toml`. Confirm `--backend modal` and that `--modal-config` is an existing host file (Compose mounts it read-only).
</Accordion>
<Accordion title="Build commit error during up">
The CLI could not resolve a 40-hex SHA. Set `SELFBENCH_BUILD_COMMIT` to the self-bench commit you intend to record on runs.
</Accordion>
<Accordion title="API or worker restarting immediately">
They connect to `temporal:7233` with no wait script. Watch `docker compose --file "$(node -p "require('path').join(require('path').dirname(require.resolve('self-bench/package.json')), 'compose.yaml')")" logs temporal` until auto-setup finishes, then confirm the API log `SelfBench API listening on http://0.0.0.0:8080`.
</Accordion>
</AccordionGroup>

Host-side `bun run dev:api` / `bun run dev:worker` are checkout-only processes. They are not the Compose stack and do not create these volumes. See [Contributing](/contributing).

## Next

<CardGroup>
<Card title="Quickstart" href="/quickstart">
Submit a tiered run against this stack and download an export.
</Card>
<Card title="Build a benchmark" href="/build-a-benchmark">
`self-bench run` flags, HEAD pin, and `--output` wait semantics.
</Card>
<Card title="Manage runs" href="/manage-runs">
`list`, `status`, `cancel`, and `download` against `SELFBENCH_API_URL`.
</Card>
<Card title="Execution backends" href="/execution-backends">
Docker vs Modal images, concurrency, limits, and Harbor pairing.
</Card>
<Card title="Deploy Temporal Cloud" href="/deploy-temporal-cloud">
Split API and worker, GCS artifacts, matching image digest and task queue.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
401s, leftover Docker resources, and export integrity failures.
</Card>
</CardGroup>