# Testing

> Full-loop UI tests via native test, headless truth drivers, effect tests, and frame-level verification without a display server dependency.

- Repository: vercel-labs/native
- GitHub: https://github.com/vercel-labs/native
- Human docs: https://grok-wiki.com/public/docs/vercel-labs-native-8ccc3580636a
- Complete Markdown: https://grok-wiki.com/public/docs/vercel-labs-native-8ccc3580636a/llms-full.txt

## Source Files

- `tests/README.md`
- `tools/linux-truth/README.md`
- `tools/windows-truth/README.md`
- `docs/src/app/testing/layout.tsx`
- `src/runtime/effects_tests.zig`

---

---
title: "Testing"
description: "Full-loop UI tests via native test, headless truth drivers, effect tests, and frame-level verification without a display server dependency."
---

`native test` is the app-directory entry point for the full-loop suite: it runs `zig build test --summary all` (or the CLI-synthesized graph for zero-config `app.zon` + `src/` apps), prints `native test: passed` on success, and refreshes `zig-out/model-contract.zon` for typed `native check`. Headless coverage rides `NullPlatform` and `TestHarness` — no window server — while live-truth loops under `tools/linux-truth` and `tools/windows-truth` drive real OS windows when host behavior must be proven.

## Testing tiers

| Tier | Surface | Display server? | What it proves |
|------|---------|-----------------|----------------|
| Full-loop unit | Generated / hand-written `src/tests.zig` | No | Markup → tree → typed dispatch → model → rebuild |
| Runtime harness | `TestHarness` + `NullPlatform` | No | Lifecycle, GPU frame present, automation commands, effects |
| Effect channel | Fake and real executors | No | Spawn/fetch/file/timer/audio requests, cancel, drain via `.wake` |
| Frame / pixel | Layout tree + CPU reference renderer | No | Frames, device-pixel presents, FNV-1a surface signatures |
| Live truth | `tools/linux-truth`, `tools/windows-truth` | Yes (Xvfb or real desktop) | Real window chrome, input, packaging, record/replay |
| Automation smoke | `native automate` against a live binary | Host-dependent | Snapshots, widget drive, deterministic screenshots |

Cross-package fixtures that do not belong to a single module live under `tests/` and still run via `zig build test`. Most suites live next to their Zig modules.

## `native test`

```sh
native test [dir]
```

Behavior (from the shared app verbs in `src/tooling/verbs.zig`):

1. Requires `app.zon` in the app directory (or pass the app path).
2. If the app owns `build.zig`, runs plain `zig build test --summary all`.
3. If the app is zero-config (`app.zon` + `src/` only), synthesizes the build graph under `.native/build/` and runs that with `--prefix` aimed at the app’s `zig-out/`.
4. On success prints: `native test: passed (test tally in the build summary above)`.

Forward `-D...` / `--release` flags the same way as `native build`. A missing framework checkout fails with `MissingFramework` and a hint to set `NATIVE_SDK_PATH` or use a `native` binary from the SDK checkout.

`native test` is also what keeps `native check` model-aware: a fresh `zig-out/model-contract.zon` lets check validate bindings, iterables, and message tags against `Model` / `Msg`. Without it, check degrades to structural validation and says so loudly.

<Tip>
Apps that own `build.zig` can emit the contract alone with `zig build model-contract`. Zero-config apps should prefer `native test`.
</Tip>

## Full-loop UI tests

Scaffolded apps get `src/tests.zig` that builds the real markup view, walks the widget tree, dispatches the same pointer messages the runtime would, and asserts on model + rebuilt view — no window.

```zig
var view = try canvas.MarkupView(Model, Msg).init(arena, main.app_markup);
var ui = canvas.Ui(Msg).init(arena);
const tree = try ui.finalize(try view.build(&ui, &model));

const plus = try expectByText(tree.root, .button, "+");
main.update(&model, tree.msgForPointer(plus.id, .up).?);
try testing.expectEqual(@as(i64, 1), model.count);

tree = try buildTree(arena, &model); // rebuild after every dispatch
```

Rules that fail tests when ignored:

| Rule | Behavior |
|------|----------|
| Tree is a snapshot | Rebuild after each `update` before the next press |
| Disabled controls | `msgForPointer` returns `null` — assert null, do not unwrap |
| Stable ids | Same control keeps its id across rebuilds while text may change |
| Markup sliders | Use `msgFor(id, .change)`; `msgForValue` is for Zig builder `on_value` only |
| Layout | `canvas.layoutWidgetTree` asserts real frames without presenting |

The scaffold helpers (`findByText` / `expectByText`) fail with a diagnostic when markup and the test drift, instead of a null-unwrap panic.

## TestHarness and NullPlatform

`TestHarness` is a heap-allocated headless driver: it embeds the multi-megabyte `Runtime`, so always use `create` / `destroy` (stack instances overflow test threads). Same heap rule applies to `UiApp` instances in tests.

```zig
const harness = try native_sdk.TestHarness().create(
    std.testing.allocator,
    .{ .size = geometry.SizeF.init(400, 300) },
);
defer harness.destroy(std.testing.allocator);

// Defaults: NullPlatform + BufferSink trace, dispatch_error_policy = .propagate
try harness.start(app); // app_start → surface_resized → frame_requested
// ... dispatch events, automation commands, effects ...
try harness.stop(app);  // app_shutdown
```

`start` installs the app and requests an initial frame. Tests that exercise degrade paths set `runtime.dispatch_error_policy` back to `.degrade` explicitly; the default is fail-loud.

`NullPlatform` records sources and presents without creating real windows. Enable GPU surface recording when asserting pixel presents:

```zig
harness.null_platform.gpu_surfaces = true;
try harness.runtime.dispatchPlatformEvent(app, .{ .gpu_surface_frame = .{
    .label = "stream-canvas",
    .size = geometry.SizeF.init(400, 300),
    .scale_factor = 1,
    .frame_index = 1,
    .timestamp_ns = 1_000_000,
    .nonblank = true,
} });
```

Framework suites use this path for bridge policy, window reconcile, automation `widget-click`, and CPU fallback when the packet presenter is unsupported (device-resolution RGBA8 present counts, width/height, scale, byte length).

## Effect tests

Side effects only leave the process through the effects channel. Tests choose an executor:

### Fake executor (deterministic, default for app tests)

```zig
app_state.effects.executor = .fake; // before first dispatch / init_fx
try app_state.dispatch(&harness.runtime, 1, .start);

const request = app_state.effects.pendingSpawnAt(0).?;
try testing.expectEqualStrings("gh", request.argv[0]);

try app_state.effects.feedLine(stream_key, "alpha");
try app_state.effects.feedExit(stream_key, 0);
// Drain the same way live platforms do: worker → wake → update
while (harness.null_platform.takeWake()) |_| {}
try harness.runtime.dispatchPlatformEvent(app, .wake);
```

Fake-side assertions cover:

- Recorded spawn / fetch / file / clipboard / timer / audio requests
- Synthetic `feedLine` / `feedExit` / `feedResponse` / `feedFileResult` / `fireTimer` / `feedAudioEvent`
- Cancel before drain: queued lines never become Msgs; exactly one `.cancelled` exit
- Slot reuse after cancel with sticky generation filtering
- Queue overflow: drop counts are loud; over-long lines truncate with flags
- Collect mode: output + stderr tails, truncation bounds, empty payloads on cancel

### Real executor (still headless)

`effects_tests.zig` also runs real processes under `TestHarness` (POSIX-oriented; many tests `SkipZigTest` on Windows). Coverage includes streaming stdout into the model, stdin + nonzero exits, cancel of a long stream, parent `HOME`/`PATH` inheritance for children, raised line bounds, collect mode, and `spawn_failed` for unspawnable binaries. Completions still drain through platform wakes — the same path production hosts use.

<Note>
Set the fake executor before `init_fx` runs if boot spawns should be recorded rather than executed.
</Note>

## Frame-level verification

Without a display server, three layers still pin visual truth:

1. **Layout frames** — `layoutWidgetTree` / `expectLayoutFrame` for geometry.
2. **Present path** — `gpu_surface_frame` events force install + present; assert retained widget text and `NullPlatform` present counters / dimensions.
3. **Reference-renderer signatures** — `referenceSurfaceSignature` (FNV-1a over RGBA8) pins catalog and theme registers:

```zig
try testing.expectEqual(
    @as(u64, 4863232662243686658),
    support.referenceSurfaceSignature(pixels),
);
```

Vector raster tests pin curvy coverage the same way (FNV-1a over the coverage grid). Theme packs (`house`, `geist`) keep separate pinned signatures so a token or emitter change fails CI instead of silently restyling.

Pinned goldens (pixel signatures, schema fingerprints, command counts) are updated deliberately: review the rendered output or counted commands first, and keep the pin comment a self-contained description of what the value represents.

For live PNG goldens at scale 1 through the deterministic CPU reference renderer, use automation screenshots (`native automate screenshot <view-label>`) — two captures of an unchanged scene are byte-identical on the same machine. See [Automation](/automation) and [Testing in CI](/testing-ci).

## Headless truth drivers

When null-platform confidence is not enough, the repo carries live-truth loops that drive showcase apps on real hosts:

### Linux (`tools/linux-truth`)

Runs under Xvfb in a container with Zig, GTK4, WebKitGTK headers. Repo mounts read-only at `/src`; builds use container-local `/work`; artifacts land in `/out`.

```sh
tools/linux-truth/run-all.sh [image|up|sync|recon|drive|suites|all]
```

| Step | Role |
|------|------|
| `image` / `up` | Build and start the long-lived container |
| `sync` | rsync host → `/work` (never write-back) |
| `recon` | Build/launch showcase apps; snapshots, inventories, both screenshot channels |
| `drive` | Replay clicks, text, wheel, resize (including min-size clamp) |
| `suites` | Engine + example suites and webview link check on real Linux |

### Windows (`tools/windows-truth`)

Drives a real unlocked Windows 11 desktop over SSH (cmd.exe default shell). Visible-desktop work hops through `schtasks /IT`; artifacts under `%TEMP%\native-truth-out\`.

```powershell
powershell -NoProfile -File tools\windows-truth\run-all.ps1 [recon|drive|effects|record|writeback|package|all]
```

Steps cover recon snapshots, interaction drive, effect stream/cancel/clipboard, record/replay with headless verification, markup write-back + hot reload, and packaged-artifact launch.

These loops answer “what does the OS actually do?” — window styles, IME, resize floors, clipboard, packaging — without trusting the null platform.

## Framework and example suite commands

From the SDK checkout (no display server for these steps unless noted):

```bash
zig build test
zig build test-desktop
zig build test-automation-protocol
zig build test-platform-info
zig build test-examples
zig build test-examples-native    # -Dplatform=null; native CLI graph for zero-config examples
zig build test-examples-mobile
zig build test-example-<name>    # e.g. test-example-notes
```

Contributor local gate (maps your diff to the suites that cover it):

```bash
scripts/gate.sh fast [ref]   # default base: main
scripts/gate.sh full         # CI-shaped local pass
```

Opt-in GUI smokes (macOS session) live under separate build steps such as `test-webview-smoke`, `test-native-shell-smoke`, and CEF variants — they use the automation protocol against real windows. Details and workflow recipes: [Testing in CI](/testing-ci).

## Verification signals

| Command / check | Success signal | Failure signal |
|-----------------|----------------|----------------|
| `native test` | Build summary + `native test: passed` | Non-zero zig exit; no pass line |
| `zig build test -Dplatform=null` | All tests green | First failing test name + expect dump |
| Fake effect test | Pending request shape + model after `.wake` | `EffectNotFound`, wrong counts, silent drops (should not happen) |
| Pixel pin | `expectEqual` on signature | Signature moved — review render before re-blessing |
| `native check` after test | Typed contract checks | “model contract: not yet built…” if artifact missing/stale |
| Linux/Windows truth | Artifacts in `/out` or `%TEMP%\native-truth-out\` | Step script non-zero; empty inventories/screenshots |

## Common failure modes

| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| Stack overflow in test thread | `TestHarness` or `UiApp` on the stack | Heap `create` / `destroy` |
| Null unwrap on press | Disabled control or stale tree | Assert null for disabled; rebuild after dispatch |
| Effects “did nothing” | Never drained wakes | `takeWake` loop + `dispatchPlatformEvent(.wake)` |
| `feedExit` after cancel | Slot already retired with `.cancelled` | Expect `error.EffectNotFound` |
| Real spawn missing PATH/HOME | Old environ wiring (regression covered) | Ensure harness uses `std.testing.environ` (default) |
| `native test` / MissingFramework | Zero-config app without SDK path | Set `NATIVE_SDK_PATH` or use checkout `zig-out/bin/native` |
| Signature golden failed | Intentional visual change or accidental token drift | Review pixels; update pin only deliberately |
| Truth loop empty screenshots | Locked desktop (Windows) or container not up (Linux) | Unlock session; `up` + `sync` before `recon` |

## Related pages

<CardGroup>
  <Card title="Testing in CI" href="/testing-ci">
    Workflows, gate scripts, Xvfb smoke, and reproducible headless checks.
  </Card>
  <Card title="Automation" href="/automation">
    Snapshots, widget driving, deterministic screenshots, record/replay.
  </Card>
  <Card title="Runtime and effects" href="/runtime-effects">
    Effect kinds, cancellation, worker wakes, and fake-executor test surface.
  </Card>
  <Card title="CLI reference" href="/cli-reference">
    `native test`, `check`, `automate`, and related flags and failure modes.
  </Card>
  <Card title="App model" href="/app-model">
    Model, Msg, update loop wiring that full-loop tests exercise.
  </Card>
  <Card title="Contributing" href="/contributing">
    Local setup, `scripts/gate.sh`, and pre-1.0 contribution expectations.
  </Card>
</CardGroup>