# Configuration reference

> SELFBENCH_* environment keys, defaults, validation, loopback token rule, artifact backends, and Temporal connection fields.

- Repository: mupt-ai/self-bench
- GitHub: https://github.com/mupt-ai/self-bench
- Human docs: https://grok-wiki.com/public/docs/mupt-ai-self-bench-62fff2be9bcc
- Complete Markdown: https://grok-wiki.com/public/docs/mupt-ai-self-bench-62fff2be9bcc/llms-full.txt

## Source Files

- `src/config.ts`
- `compose.yaml`
- `docs/operations.md`
- `src/temporal/worker-main.ts`
- `src/modal-auth.ts`

---

---
title: "Configuration reference"
description: "SELFBENCH_* environment keys, defaults, validation, loopback token rule, artifact backends, and Temporal connection fields."
---

`loadConfig()` parses `process.env` once at process start for `node dist/api-main.js`, `node dist/temporal/worker-main.js`, `self-bench-repair`, and `self-bench-reaudit`. It validates `SELFBENCH_*` keys with Zod, applies defaults, enforces the loopback token rule, and returns a typed `SelfBenchConfig` with `apiHost` / `apiPort` / `apiToken`, `artifact` (`local` or `gcs`), `execution` (`docker` or `modal`), `harborEnvironment`, `activityConcurrency`, optional `buildCommit`, and `temporal`. Extra process-environment keys are ignored. The `self-bench` CLI does **not** call `loadConfig()`; it is an HTTP client that reads `SELFBENCH_API_URL` and `SELFBENCH_API_TOKEN`.

```mermaid
flowchart TB
  subgraph cli ["self-bench CLI"]
    APIURL["SELFBENCH_API_URL"]
    CLITOKEN["SELFBENCH_API_TOKEN as Bearer"]
    UP["up writes backend + modal mount"]
  end
  subgraph consumers ["loadConfig() consumers"]
    API["api-main"]
    WRK["worker-main"]
    COMP["repair / reaudit"]
  end
  subgraph cfg ["SelfBenchConfig"]
    LISTEN["apiHost / apiPort / apiToken"]
    ART["artifact: local | gcs"]
    EX["execution: docker | modal"]
    HB["harborEnvironment"]
    TMP["temporal address / namespace / tls / apiKey / taskQueue"]
  end
  subgraph secrets ["Worker secrets outside the schema"]
    OA["OPENAI_API_KEY"]
    GH["GH_TOKEN"]
    MD["MODAL_TOKEN_ID / MODAL_TOKEN_SECRET"]
    SUB["SELFBENCH_PI_AUTH_JSON / SELFBENCH_CODEX_AUTH_JSON"]
  end
  CLI -->|HTTP| API
  API --> cfg
  WRK --> cfg
  COMP --> cfg
  WRK --> secrets
  UP --> WRK
```

## Who reads which keys

| Process | Entry | Parser | Role |
| --- | --- | --- | --- |
| API | `node dist/api-main.js` (image default `CMD`) | `loadConfig()` | Bind address, Bearer token, Temporal **client**, artifact store |
| Worker | `node dist/temporal/worker-main.js` | `loadConfig()` after empty Modal-token cleanup | Temporal **worker**, sandboxes, Harbor `--env`, artifact store |
| `self-bench-repair` / `self-bench-reaudit` | companion CLIs | `loadConfig()` | Same execution backend as the worker; no Temporal listen |
| `self-bench` | `dist/cli.js` | raw `process.env` | `SELFBENCH_API_URL`, client `SELFBENCH_API_TOKEN`, `up` overlay |
| `self-bench-eval` / `self-bench-validate` / `self-bench-agent-smoke` | companion CLIs | flags + auth env | Do **not** call `loadConfig()` |

`bun run dev:api`, `bun run start:api`, `bun run dev:worker`, and `bun run start:worker` use the same parsers as the packaged Node entrypoints.

## `SELFBENCH_*` keys parsed by `loadConfig()`

Unknown keys on `process.env` are stripped. Empty string is **not** treated as unset except for `SELFBENCH_BUILD_COMMIT` (preprocessed to missing) and the worker's Modal token cleanup (not part of this schema).

### API listen and token

<ParamField body="SELFBENCH_API_HOST" type="string" default="127.0.0.1">
Bind address passed to `server.listen`. Compose sets `0.0.0.0` inside the container.
</ParamField>

<ParamField body="SELFBENCH_API_PORT" type="integer" default="8080">
TCP port. Coerced integer in `1..65535`. Compose publishes `127.0.0.1:8080:8080`.
</ParamField>

<ParamField body="SELFBENCH_API_TOKEN" type="string" optional>
Non-empty Bearer secret. Optional only on loopback. Omitted from `SelfBenchConfig` when unset. Compose default is `selfbench-local` via `${SELFBENCH_API_TOKEN:-selfbench-local}`.
</ParamField>

<ParamField body="SELFBENCH_BUILD_COMMIT" type="string" optional>
Full 40-hex commit SHA. Empty string is treated as unset. Stored lowercased as `buildCommit`. When set, the API prefers it over the run body's `selfbenchCommit`.
</ParamField>

### Artifact backend

<ParamField body="SELFBENCH_ARTIFACT_BACKEND" type="local | gcs" default="local">
Selects `LocalArtifactStore` or `GcsArtifactStore`. Compose forces `local`.
</ParamField>

<ParamField body="SELFBENCH_ARTIFACT_DIR" type="string" default=".selfbench/artifacts">
Root directory for `local`. Resolved to an absolute path. Compose uses `/var/lib/selfbench/artifacts` on the named `artifacts` volume.
</ParamField>

<ParamField body="SELFBENCH_GCS_BUCKET" type="string" optional>
Required when the backend is `gcs`. Startup throws `SELFBENCH_GCS_BUCKET is required for GCS`.
</ParamField>

<ParamField body="SELFBENCH_GCS_PREFIX" type="string" default="selfbench">
Object-name prefix. Leading and trailing slashes are stripped. Reads reject URIs outside `gs://<bucket>/<prefix>/…`.
</ParamField>

### Execution and Harbor

<ParamField body="SELFBENCH_EXECUTION_BACKEND" type="docker | modal" default="docker">
Sandbox executor. `self-bench up --backend` writes this for Compose.
</ParamField>

<ParamField body="SELFBENCH_HARBOR_ENVIRONMENT" type="docker | modal" optional>
Harbor `--env` for nop/oracle gates. Defaults to `SELFBENCH_EXECUTION_BACKEND`. `self-bench up` always sets both to the same value.
</ParamField>

<ParamField body="SELFBENCH_DOCKER_IMAGE" type="string" default="selfbench-sandbox:local">
Image for Docker sandboxes. `self-bench up --backend docker` builds `Dockerfile.sandbox` to this tag.
</ParamField>

<ParamField body="SELFBENCH_MODAL_APP" type="string" default="selfbench">
Modal app name (`apps.fromName`, create-if-missing).
</ParamField>

<ParamField body="SELFBENCH_MODAL_ENVIRONMENT" type="string" optional>
Modal environment name for the app and ephemeral secrets.
</ParamField>

<ParamField body="SELFBENCH_MODAL_IMAGE" type="string" default="node:22-bookworm">
Registry image Modal extends with `gh`, Pi, and Codex.
</ParamField>

<ParamField body="SELFBENCH_ACTIVITY_CONCURRENCY" type="integer" optional>
`maxConcurrentActivityTaskExecutions`, integer `1..100`. When unset: `1` for Docker, `20` for Modal.
</ParamField>

### Temporal connection

<ParamField body="SELFBENCH_TEMPORAL_ADDRESS" type="string" default="127.0.0.1:7233">
gRPC target. Compose sets `temporal:7233`. Published on the host as `127.0.0.1:7233`.
</ParamField>

<ParamField body="SELFBENCH_TEMPORAL_NAMESPACE" type="string" default="default">
Namespace for the Temporal client, worker, and workflow start.
</ParamField>

<ParamField body="SELFBENCH_TASK_QUEUE" type="string" default="selfbench-dev">
Shared task queue. API `workflow.start` and the worker poller must use the same value.
</ParamField>

<ParamField body="SELFBENCH_TEMPORAL_TLS" type='"true" | "false"' default="false">
String enum only (`true` / `false`). Transformed to a boolean. Missing env becomes `false`. Case-sensitive: `TRUE` fails parse.
</ParamField>

<ParamField body="SELFBENCH_TEMPORAL_API_KEY" type="string" optional>
Passed to Temporal `Connection.connect` / `NativeConnection.connect` only when non-empty.
</ParamField>

## Keys outside `loadConfig()`

These names appear in Compose, the CLI, or workers but are not fields on `SelfBenchConfig`.

| Key | Default | Used by |
| --- | --- | --- |
| `SELFBENCH_API_URL` | `http://127.0.0.1:8080` | CLI HTTP client (`list`, `status`, `run`, `cancel`, `download`) |
| `SELFBENCH_MODAL_CONFIG_PATH` | `/dev/null` in Compose; `~/.modal.toml` from `self-bench up --backend modal` | Host bind-mount onto `/root/.modal.toml` (read-only) |
| `DOCKER_GID` | `0` | Compose `group_add` so the worker can use the Docker socket |
| `OPENAI_API_KEY` | unset | Worker sandboxes; preferred over subscription JSON |
| `SELFBENCH_PI_AUTH_JSON` | else `~/.pi/agent/auth.json` | Pi / review / discovery / authoring fallback |
| `SELFBENCH_CODEX_AUTH_JSON` | else `CODEX_AUTH_JSON_PATH` or `~/.codex/auth.json` | Codex / coupling-repair fallback |
| `CODEX_AUTH_JSON_PATH` | `~/.codex/auth.json` | File fallback for Codex subscription auth |
| `GH_TOKEN` | else `gh auth token` | Worker GitHub reads; Compose passes it only to the worker |
| `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` | unset | Modal SDK; empty values are deleted at worker start |

<Note>
`self-bench-eval`, `self-bench-validate`, and `self-bench-agent-smoke` take `--environment docker|modal` on the command line (eval/validate default `modal`). They do not read `SELFBENCH_EXECUTION_BACKEND`.
</Note>

## Loopback token rule

After Zod parse, `loadConfig()` requires a token unless the host is one of these **exact** strings:

- `127.0.0.1`
- `::1`
- `localhost`

Any other host — including Compose's `0.0.0.0` — throws:

```text
SELFBENCH_API_TOKEN is required when the API listens beyond loopback
```

An empty `SELFBENCH_API_TOKEN` fails Zod (`min(1)`) before this check. Compose's `${SELFBENCH_API_TOKEN:-selfbench-local}` substitutes the local default when the variable is unset or empty.

When a token is present, every route except `GET /healthz`, `GET /`, and `GET /assets/*` requires `Authorization: Bearer <token>` compared with `timingSafeEqual`. When the token is omitted on loopback, `authorized()` returns true and no Bearer header is required. The CLI sends `Authorization` only when `SELFBENCH_API_TOKEN` is a non-empty string.

<Warning>
The Compose API binds `0.0.0.0` inside the container, so the local stack always needs a token. Export the same value in the CLI shell. The published port is still `127.0.0.1:8080` on the host.
</Warning>

## Validation and derived defaults

| Check | Failure |
| --- | --- |
| Enum / port / concurrency / SHA / TLS parse | Zod error; process exits at `loadConfig()` |
| Host not loopback and no token | `SELFBENCH_API_TOKEN is required when the API listens beyond loopback` |
| `SELFBENCH_ARTIFACT_BACKEND=gcs` without bucket | `SELFBENCH_GCS_BUCKET is required for GCS` |
| `SELFBENCH_BUILD_COMMIT` set but not 40 hex | Zod regex failure (runtime) or `SELFBENCH_BUILD_COMMIT must be a full commit SHA` (image build script) |
| CLI has no baked SHA and no git | `SelfBench was built without commit metadata; set SELFBENCH_BUILD_COMMIT to a full commit SHA` |
| `self-bench up --backend` not `docker` or `modal` | `--backend must be "docker" or "modal"` |
| `--modal-config` without Modal backend | `--modal-config requires --backend modal` |

Derived after a successful parse:

| Field | Rule |
| --- | --- |
| `activityConcurrency` | Env override, else `20` when execution is `modal`, else `1` |
| `harborEnvironment` | Env override, else the execution backend |
| `buildCommit` | Lowercased SHA, omitted when unset |
| `execution` | `{ kind: "docker", image }` or `{ kind: "modal", app, image, environment? }` |
| `artifact` | `{ kind: "local", directory }` or `{ kind: "gcs", bucket, prefix }` |
| `temporal.apiKey` / `apiToken` | Omitted when the env value is empty |

The worker then:

1. Calls `removeEmptyModalCredentialOverrides()` so blank `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` cannot hide a mounted `~/.modal.toml`.
2. If `execution.kind === "docker"` **or** `harborEnvironment === "docker"`, runs `docker info` and `docker compose version` (30s each) before connecting to Temporal.
3. Creates the worker with `maxConcurrentActivityTaskExecutions: config.activityConcurrency`.
4. Logs `SelfBench worker polling <namespace>/<taskQueue> with activity concurrency <n>`.

The API logs `SelfBench API listening on http://<host>:<port>` after `listen` succeeds.

## Artifact backends

`createArtifactStore(config.artifact)` is shared by the API (provenance upload, export download) and the worker (run artifacts). Writes are create-only: local uses `wx` / hard-link, GCS uses `ifGenerationMatch: 0`. An identical SHA-256 + size is treated as idempotent; different contents throw `artifact already exists with different contents`.

Keys must be non-empty, must not start with `/`, and must not contain a `..` path segment (`unsafe artifact key`).

### Local

- Root is `resolve(SELFBENCH_ARTIFACT_DIR)`.
- Object path is `root/key`; reads reject any resolved path outside that root (`artifact path escapes root`).
- URI shape: `file://<absolute-path>`.
- Compose mounts the `artifacts` volume at `/var/lib/selfbench/artifacts` on **both** API and worker.

### GCS

- Client is `new Storage()` — the process Application Default Credentials, not a `SELFBENCH_*` key.
- Object name is `<prefix>/<key>` when the prefix is non-empty.
- URI shape: `gs://<bucket>/<object>`.
- `get` / `openRead` reject a different bucket or a path outside the configured prefix (`artifact is outside configured bucket`).
- Custom metadata stores `sha256`; streamed reads also check size and digest (`artifact integrity check failed`).

Exports are unencrypted. Confine IAM to the configured prefix independently of the application prefix check.

## Temporal connection fields

`connectTemporalClient` (API) and `connectTemporalWorker` (worker) pass the same object:

```ts
{
  address: config.temporal.address,
  tls: config.temporal.tls,
  ...(config.temporal.apiKey ? { apiKey: config.temporal.apiKey } : {}),
}
```

There is no extra check that TLS is on when an API key is set. The API `Client` uses `namespace`. `workflow.start` uses `taskQueue` and a 14-day `workflowExecutionTimeout`. The worker uses the same `namespace` and `taskQueue`.

<Warning>
API and worker must share image digest, `SELFBENCH_TASK_QUEUE`, Temporal address/namespace/TLS/API key, and artifact backend. A queue mismatch silently parks workflows with no poller.
</Warning>

Compose Temporal is `temporalio/auto-setup:1.29.6` on Postgres 17 with development credentials `temporal` / `temporal`. State lives in the `selfbench_temporal-postgres` volume. Self-bench has no application database migrations.

## Compose overlay versus schema defaults

`self-bench up` runs `docker compose --file compose.yaml up -d --build` after writing:

| Variable | Docker backend | Modal backend |
| --- | --- | --- |
| `SELFBENCH_EXECUTION_BACKEND` | `docker` | `modal` |
| `SELFBENCH_HARBOR_ENVIRONMENT` | `docker` | `modal` |
| `SELFBENCH_BUILD_COMMIT` | resolved 40-hex SHA | same |
| `SELFBENCH_MODAL_CONFIG_PATH` | unset (Compose falls back to `/dev/null`) | `--modal-config` or `~/.modal.toml` |

Compose then overlays schema defaults:

```text
SELFBENCH_API_HOST=0.0.0.0
SELFBENCH_API_TOKEN=${SELFBENCH_API_TOKEN:-selfbench-local}
SELFBENCH_TEMPORAL_ADDRESS=temporal:7233
SELFBENCH_ARTIFACT_BACKEND=local
SELFBENCH_ARTIFACT_DIR=/var/lib/selfbench/artifacts
SELFBENCH_DOCKER_IMAGE=selfbench-sandbox:local
```

Worker-only Compose environment: `GH_TOKEN`, `OPENAI_API_KEY`, `MODAL_TOKEN_ID`, `MODAL_TOKEN_SECRET`, `SELFBENCH_PI_AUTH_JSON`, `SELFBENCH_CODEX_AUTH_JSON`. The worker runs as `root`, adds `DOCKER_GID`, and mounts `/var/run/docker.sock`. The API image user is `node` and never receives the Docker socket or model credentials.

## Example environments

<CodeGroup>

```bash title="Schema defaults (no Compose)"
# API: http://127.0.0.1:8080 — token optional
# artifacts: .selfbench/artifacts
# Temporal: 127.0.0.1:7233 / default / selfbench-dev / tls=false
# execution: docker / selfbench-sandbox:local / concurrency 1
```

```bash title="Local Compose (self-bench up)"
export OPENAI_API_KEY=...
export GH_TOKEN="$(gh auth token)"
export SELFBENCH_API_TOKEN="$(openssl rand -hex 24)"
export SELFBENCH_API_URL=http://127.0.0.1:8080
self-bench up --backend modal
```

```bash title="Split API + worker (Temporal Cloud + GCS + Modal)"
# API process
SELFBENCH_API_HOST=0.0.0.0
SELFBENCH_API_TOKEN=...
SELFBENCH_ARTIFACT_BACKEND=gcs
SELFBENCH_GCS_BUCKET=...
SELFBENCH_GCS_PREFIX=selfbench
SELFBENCH_TEMPORAL_ADDRESS=...
SELFBENCH_TEMPORAL_NAMESPACE=...
SELFBENCH_TEMPORAL_API_KEY=...
SELFBENCH_TEMPORAL_TLS=true
SELFBENCH_TASK_QUEUE=selfbench-prod

# Worker process (same Temporal + GCS + queue + image digest)
SELFBENCH_EXECUTION_BACKEND=modal
SELFBENCH_HARBOR_ENVIRONMENT=modal
MODAL_TOKEN_ID=...
MODAL_TOKEN_SECRET=...
GH_TOKEN=...
OPENAI_API_KEY=...
```

</CodeGroup>

The worker owns model, GitHub, Modal, and Harbor credentials. Keep API and worker service accounts separate.

## Sandbox-injected `SELFBENCH_*` keys

The worker writes these into sandbox processes. They are not operator configuration.

| Key | Purpose |
| --- | --- |
| `SELFBENCH_DISCOVERY_OUTPUT` | `/work/discovery.json` |
| `SELFBENCH_TASK_OUTPUT` | `/work/tasks` |
| `SELFBENCH_REVIEW_OUTPUT` | `/work/review.json` |
| `SELFBENCH_REPAIR_MODEL` | Authoring / repair model name (companion default `gpt-5.6-sol`) |
| `SELFBENCH_PI_AUTH_JSON` / `SELFBENCH_CODEX_AUTH_JSON` | Stage-specific subscription material when no `OPENAI_API_KEY` |

Sandboxes also receive `OPENAI_API_KEY` and `GH_TOKEN` when those credentials exist on the worker. Bootstrap writes Pi auth to `$HOME/.pi/agent/auth.json` and a GitHub `x-access-token` credential helper, then deletes both on exit.

<AccordionGroup>
<Accordion title="Failure modes">

- **API exits at startup on Compose without a token** — host is `0.0.0.0`. Set `SELFBENCH_API_TOKEN` or accept `selfbench-local`.
- **CLI `401 unauthorized`** — CLI token missing or different from the API token.
- **Workflows stay queued** — API and worker `SELFBENCH_TASK_QUEUE` or Temporal namespace/address differ.
- **GCS startup throw** — backend `gcs` without `SELFBENCH_GCS_BUCKET`.
- **Modal profile ignored** — empty `MODAL_TOKEN_*` in Compose; the worker now deletes those empties. Non-empty invalid tokens still override the mount.
- **Worker refuses to start** — Docker backend or Docker Harbor without a working `docker` / `docker compose` on the worker.
- **CLI cannot resolve commit** — published package without git and without `SELFBENCH_BUILD_COMMIT`.

</Accordion>
</AccordionGroup>

## Next

<CardGroup>
<Card title="Installation" href="/installation">
Prerequisites and the first environment exports before `self-bench up`.
</Card>
<Card title="Start the local stack" href="/start-local-stack">
How `up` / `down` apply the Compose overlay, backends, and volumes.
</Card>
<Card title="Deploy Temporal Cloud" href="/deploy-temporal-cloud">
Split API and worker, GCS, TLS, API key, and matching task queue.
</Card>
<Card title="Authentication" href="/authentication">
`OPENAI_API_KEY` precedence, subscription JSON, `GH_TOKEN`, Bearer, Modal profile versus tokens.
</Card>
<Card title="Execution backends" href="/execution-backends">
Docker versus Modal images, concurrency, and Harbor pairing.
</Card>
<Card title="HTTP API" href="/http-api">
Which routes honor the Bearer token and how provenance/export use the artifact store.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
Export 409, SHA-256 mismatch, leftover Docker resources.
</Card>
</CardGroup>