# Testing

> How to run the test suites, what they cover, fixtures, and how continuous integration executes them.

- Repository: egoist/lorca
- GitHub: https://github.com/egoist/lorca
- Human docs: https://grok-wiki.com/public/docs/egoist-lorca-2cf67495e5e6
- Complete Markdown: https://grok-wiki.com/public/docs/egoist-lorca-2cf67495e5e6/llms-full.txt

## Source Files

- `package.json`
- `Cargo.toml`
- `.github/workflows/release-cli.yml`
- `crates/cli/tests/provider_settings.rs`
- `macos/Package.swift`

---

---
title: "Testing"
description: "How to run the test suites, what they cover, fixtures, and how continuous integration executes them."
---

Run `cargo test` from the repo root for the Rust workspace (agent loop, crypto, SSE, CLI, relay, and related crates). That is the default local gate named in the root README.

```bash
cargo test
```

Success: every package reports `ok`, including the `lorca` binary integration tests under `crates/cli/tests/`.

## Suites at a glance

| Surface | Command | Runtime | What it proves |
| --- | --- | --- | --- |
| Rust workspace | `cargo test` | ~minutes on a cold cache | Unit + async tests across `crates/*`, plus CLI integration tests |
| Phone core | `bun run --cwd mobile test` | about 1 command | Pairing/model/format helpers and reading/engine behavior with native bridges mocked |
| Mac startup | `bun run test:mac-startup` | about 1 command | CLI launcher concurrency without loading AppKit |
| Mac SPM | `cd macos && swift test` | SPM `LorcaTests` | Notifications, transcript accessibility, markdown layout, running tasks |
| CLI release CI | tag `cli-v*` or `workflow_dispatch` | GitHub Actions | Cross-platform release builds + install-script verification (not `cargo test`) |

<Note>
There is one workflow under `.github/workflows/`: `release-cli.yml`. It does not run the unit/integration suites above. Local `cargo test` / Bun / Swift runs are the coverage for those.
</Note>

## Run Rust tests

<Steps>
  <Step title="Default workspace run">
    From the repo root:

    ```bash
    cargo test
    ```

    Workspace members: `lorca-agent`, `lorca-provider-auth`, `lorca-relay`, `lorca`, `lorca-mobile`, `lorca-markdown` (`Cargo.toml`). The `lorca` binary requires the `cli` feature (default on).
  </Step>
  <Step title="Package filters">
    ```bash
    cargo test -p lorca
    cargo test -p lorca-relay
    cargo test -p lorca-agent
    cargo test -p lorca-provider-auth
    cargo test -p lorca-markdown
    ```
  </Step>
  <Step title="Optional Postgres for relay DB">
    SQLite always runs (temp file). Postgres joins when this is set:

    ```bash
    export LORCA_RELAY_TEST_POSTGRES='postgres://postgres:lorca@127.0.0.1:55432/lorca'
    cargo test -p lorca-relay
    ```

    Identities are random so shared databases are safe across concurrent runs.
  </Step>
</Steps>

### CLI integration tests

Under `crates/cli/tests/`:

| File | Behavior under test |
| --- | --- |
| `provider_settings.rs` | `providers.api_key` returns only the requested key/base URL; secrets stay out of `snapshot()` and credential statuses; unknown kinds and missing `kind` error |
| `serve_ready.rs` | `lorca serve --ready-stdout` prints one JSON `{"event":"ready","port":…}` with logs off; WebSocket `/ws` answers `bootstrap`; a failed bind exits with empty stdout and no ready line |

`serve_ready.rs` is gated on `feature = "cli"` and spawns `CARGO_BIN_EXE_lorca` with a cleared env (`RUST_LOG=off`), temp `--home`, and `--port`.

### Relay coverage (high signal)

- **DB store** (`crates/relay/src/db/tests.rs`): slots (first/latest), refused puts, deleted groups, byte-budget pages, machines/challenges/revocation, pairing mailbox, presence/events, sweeps, orphans, inactive identities, usage recount, backward group paging.
- **HTTP routes** (`crates/relay/src/routes/tests.rs`): encrypted binary file round-trip and idempotent PUT; auth/metadata/quota/missing objects; envelope overhead and chunked body caps.
- **In-module unit tests** also sit in `hub`, `store`, `sweep`, `metrics`, `push`, and `main`.

Route tests stand up an in-process axum server on `127.0.0.1:0`, local SQLite + file store under a temp home, and tear both down on drop.

### Agent and provider unit tests

Most agent behavior is unit-tested inside `crates/agent` (retry, compaction, login shell, SSE/JSON, providers, harness, tools, agent loop). Provider-auth OAuth helpers are covered in `crates/provider-auth`.

## Live provider tests (ignored)

Three DeepSeek live tests are `#[ignore]` and need a real key:

```bash
DEEPSEEK_API_KEY=… cargo test -p lorca-agent live_deepseek -- --ignored --nocapture
DEEPSEEK_API_KEY=… cargo test -p lorca-agent live_deepseek_takes -- --ignored --nocapture
DEEPSEEK_API_KEY=… cargo test -p lorca-agent live_deepseek_summary -- --ignored --nocapture
```

| Filter | Location | Intent |
| --- | --- | --- |
| `live_deepseek_search_then_tool_call_replays` | Anthropic-compatible DeepSeek path | Search then tool call; seals/server blocks replay |
| `live_deepseek_takes_a_turns_four_cache_marks` | same | Cache-mark behavior across a turn |
| `live_deepseek_summary` | compaction | Live summary path |

Without `DEEPSEEK_API_KEY`, those bodies return early. They are not part of a plain `cargo test`.

## Run mobile tests

```bash
bun run --cwd mobile test
# equivalent: cd mobile && bun test src
```

| File | Coverage |
| --- | --- |
| `mobile/src/core/core.test.ts` | Pairing URL parse/reject; sent-message and command-card rules; provider kind/API-key/defaults/models; attachment summaries; dictation join; sidebar-style previews |
| `mobile/src/core/reading.test.ts` | Real engine/store with mocked RN/Expo/native core: read acknowledgements, background vs foreground unread, notifications, working-row states, snapshot/event ordering, running tasks |

Related check (not a test runner): `bun run --cwd mobile typecheck` (`tsc --noEmit`).

<Info>
Under Bun there is no native localization module, so i18n falls back to English the same way the phone core comments describe for these tests.
</Info>

## Run macOS tests

### Startup concurrency (no AppKit)

```bash
bun run test:mac-startup
```

`scripts/test-mac-startup.ts` compiles `macos/Tests/LaunchConcurrency.swift` with production `CLILaunchWorker.swift` and `StartupTrace.swift`, runs the binary against a temp directory, then deletes it. It checks launcher progress with the main thread blocked, fragmented readiness, early exit, and cancel without leaving a child.

### SPM `LorcaTests`

```bash
cd macos && swift test
```

`Package.swift` defines target `LorcaTests` at `Tests/Notifications`, depending on the `Lorca` executable target.

| File | Themes |
| --- | --- |
| `ChatNotificationTests.swift` | Failure vs recovered reply notifications; pending confirmation; suppress when read/visible |
| `ChatRowRunTests.swift` | Working row vs reply/status ordering; paging and removals |
| `TranscriptAccessibilityTests.swift` | Accessibility elements for on-screen rows, paging identity, selection/scroll |
| `MarkdownLayoutTests.swift` | Code-block padding, soft breaks, list indent, measured width |
| `RunningTaskTests.swift` | Elapsed time, status text, terminal-only running tasks |

## Fixtures and isolation patterns

```text
temp home / temp DB
  ├─ CLI tests:   $TMPDIR/lorca-provider-settings-<uuid>
  │               $TMPDIR/lorca-ready-<uuid>  (+ Drop cleanup)
  ├─ Relay routes: $TMPDIR/lorca-binary-<uuid>/{relay.db,files}
  ├─ Relay DB:     $TMPDIR/lorca-relay-test-<uuid>.db
  └─ Mac startup:  $TMPDIR/lorca-launch-tests-*  (mkdtemp, always removed)
```

Common patterns:

- **Temp home + Drop** clears directories even when a test fails.
- **Cleared process env** on `serve` children (`env_clear`, then `RUST_LOG=off`) so host credentials do not leak into the binary under test.
- **In-process servers** bind `127.0.0.1:0`; no fixed ports.
- **Mobile mocks** replace `react-native`, Expo modules, and `modules/lorca-core` so no account or provider is contacted.

## Continuous integration

Workflow: `.github/workflows/release-cli.yml` (`Release CLI`).

| Trigger | Effect |
| --- | --- |
| Push tag `cli-v*` | Build → install-script test → draft GitHub release |
| `workflow_dispatch` | Same path on the selected commit |

Jobs in order:

1. **`version`** — reads `lorca` version from `cargo metadata`; expects tag `cli-v<version>`; refuses tag/version/commit mismatches; skips work if that release is already published.
2. **`build`** (matrix) — `cargo build` / `cargo zigbuild --release --locked -p lorca` for `macos-aarch64`, `linux-aarch64` (musl), `linux-x86_64` (musl), `windows-x86_64` (gnu via zig).
3. **`test`** (matrix: `ubuntu-latest`, `macos-latest`, `windows-latest`) — serves built archives on `http://127.0.0.1:8000`, sets `LORCA_DOWNLOAD_URL`, runs install scripts twice (install then update), asserts `lorca --version` equals `lorca <version>`, and on Windows checks user PATH contains `\.local\bin`.
4. **`publish`** — drafts or refreshes GitHub release `cli-v<version>` with archives (does not auto-publish).

Concurrency group `release-cli` keeps one run at a time. `CARGO_TERM_COLOR=always`. No secrets required for drafting; the job token writes the release.

<Warning>
CI here validates **release artifacts and install scripts**, not the Rust/mobile/mac unit suites. Before tagging a CLI release, run `cargo test` locally when you changed crates.
</Warning>

## Surfaces without automated suites in-repo

| Area | Status |
| --- | --- |
| `web/` (TanStack Start site) | No `test` script or `*.test.*` / `*.spec.*` files in tree |
| Full AppKit UI | Exercised via SPM notification/transcript tests and manual `bun run dev`; startup concurrency is the automated launcher check |
| End-to-end phone ↔ relay ↔ Runner | Not a single checked-in E2E runner; relay/CLI/mobile cover pieces separately |

## Troubleshooting

| Symptom | Cause | Fix |
| --- | --- | --- |
| `serve_ready` times out on ready line | Binary hung or logs interfered | Confirm default `cli` feature; tests already force `RUST_LOG=off` and `--ready-stdout` |
| Relay DB tests skip Postgres behavior | `LORCA_RELAY_TEST_POSTGRES` unset | Export a reachable `postgres://…` URL, then re-run `-p lorca-relay` |
| `live_deepseek_*` never runs | Ignored + no key | Pass `--ignored` and set `DEEPSEEK_API_KEY` |
| `test:mac-startup` compile fails | Missing `swiftc` / Xcode CLT | Install Xcode command-line tools; run on macOS |
| Release workflow “already published” | Tag release exists non-draft | Delete published release + tag before rebuilding, or bump `[workspace.package].version` |

## Related pages

<CardGroup>
  <Card title="Installation" href="/installation">
    Prerequisites and first successful install before running suites.
  </Card>
  <Card title="CLI reference" href="/cli-reference">
    `serve`, flags such as `--ready-stdout` / `--home` / `--port`, and exit behavior used by integration tests.
  </Card>
  <Card title="crates/relay reference" href="/ref-crates-relay">
    Store backends, routes, and the Postgres test env var surface.
  </Card>
  <Card title="Deployment and operations" href="/deployment">
    Release builds, install scripts, and how CI drafts CLI archives.
  </Card>
</CardGroup>

Next: run `cargo test` from the repo root and confirm the workspace finishes with all packages `ok`.