# Integration testing
> How `packages/integration-tests` boots the backend and gatekeepers as real workers in workerd via `createTestHarness()` and speaks Cap'n Web over the same `/api` WebSocket the browser uses. Documents why fake timers cannot work out-of-process, why a fixture gatekeeper covers overseer logic, the ~3s `server.reset()` cost and convention-based isolation, and the pluggable network interceptor.
- Repository: cloudflare/cloudflare-os
- GitHub: https://github.com/cloudflare/cloudflare-os
- Human docs: https://grok-wiki.com/public/docs/cloudflare-cloudflare-os-838773bb92dd
- Complete Markdown: https://grok-wiki.com/public/docs/cloudflare-cloudflare-os-838773bb92dd/llms-full.txt
## Source Files
- `docs/integration-testing.md`
- `packages/integration-tests/src/harness.ts`
- `packages/integration-tests/src/network-interceptor.ts`
- `packages/integration-tests/src/rpc-client.ts`
- `packages/integration-tests/fixtures/gatekeeper-test/src/test-gatekeeper.ts`
- `packages/integration-tests/__tests__/observer-reverification.test.ts`
---
---
title: "Integration testing"
description: "How `packages/integration-tests` boots the backend and gatekeepers as real workers in workerd via `createTestHarness()` and speaks Cap'n Web over the same `/api` WebSocket the browser uses. Documents why fake timers cannot work out-of-process, why a fixture gatekeeper covers overseer logic, the ~3s `server.reset()` cost and convention-based isolation, and the pluggable network interceptor."
---
`packages/integration-tests` runs `workshop-backend` and one or more gatekeepers as real Workers in workerd, booted through wrangler's `createTestHarness()` with their checked-in `wrangler.jsonc` patched in memory. Tests connect a Cap'n Web session over a WebSocket to `/api` — the same transport the browser uses — and serve an `ObserverConfigCallback` the overseer calls back into. Nothing is stubbed except outbound HTTP, which means the code under test runs in another process; most of the suite's design follows from that.
## Two kinds of suite
| | this repo's `packages/integration-tests` | a consumer repo's per-vendor suite |
|---|---|---|
| Runs | `pnpm test` (part of CI's normal test job) | its own CI step |
| Gatekeeper | a fixture Worker whose verification outcome the tests set | a real vendor gatekeeper, unmodified |
| Covers | the overseer's observer logic | a genuinely expired credential, end to end |
| Owns | the harness, interceptor, and RPC client | that vendor's handlers and token minting |
A consumer repo vendors this repo as a `public/` submodule and consumes the toolkit as a workspace dependency (`public/packages/integration-tests` in its `pnpm-workspace.yaml`). No such suite lives in this repo, and nothing here depends on one existing — but the harness takes a *list* of gatekeepers and the interceptor takes *pluggable* handler modules precisely so that suite can be added outside this repo without forking either.
## Harness topology
```text
vitest process (Node) workerd (createTestHarness)
───────────────────── ───────────────────────────
rpc-client.ts ── ws://…/api ────────────▶ workshop-backend (primary)
connect() / stubFor() │ GATEKEEPER_TEST
ObserverConfigRecorder ◀── callback ─────┤ (service binding,
│ entrypoint GatekeeperVendor)
harness.fetchWorker(…) ─────────────────▶ gatekeeper-test (fixture)
│ /control/verify-outcome
network-interceptor.ts │ /control/ambient-verification-count
globalThis.fetch patched ◀── outbound fetch() routed back to Node
```
### `startHarness()`
`src/harness.ts` reads each checked-in `wrangler.jsonc` with `jsonc-parser`, validates the fields it touches against a loose Zod schema (`WORKER_CONFIG`), and hands the result to `createTestHarness()` as an inline config.
Each spec is `{ binding, dir, patch? }`. `binding` is the service-binding suffix: `GATEKEEPER_` is what the Workshop scans for, and it lowercases the suffix into the vendor id — `"JIRA"` here is vendor id `"jira"` in every RPC. `dir` holds the `wrangler.jsonc` to boot. `patch` adjusts the parsed config before boot.
Adjust `workshop-backend`'s config after the harness's own rewrites.
Harness `root`. Override when a gatekeeper lives outside this repo.
`startTestGatekeeperHarness()` is the one-liner used by the in-repo suites: it boots the Workshop with only the bundled fixture gatekeeper bound.
```ts
// packages/integration-tests/src/harness.ts
export function startTestGatekeeperHarness(): Promise {
return startHarness({
gatekeepers: [{ binding: TEST_GATEKEEPER_BINDING, dir: TEST_GATEKEEPER_DIR }],
});
}
```
Exported constants:
| Constant | Value / meaning |
|---|---|
| `TEST_GATEKEEPER_DIR` | `fixtures/gatekeeper-test` |
| `TEST_GATEKEEPER_WORKER` | `"gatekeeper-test"` |
| `TEST_GATEKEEPER_BINDING` | `"TEST"` |
| `TEST_VENDOR_ID` | `"test"` (lowercased binding) |
| `ADMIN_USERNAME` | `"admin"` — the username `vars.ADMINS` grants deployment-admin rights to, mirroring `run-dev-server.js` |
### Config rewrites the harness performs
`build.cwd` is pinned to the worker's own directory — a worker whose `main` is generated (capnweb-validate) otherwise emits output in the wrong place, the same reason `run-dev-server.js` pins it. `main` is then made absolute: an inline config has no file path of its own, so wrangler would resolve a relative `main` against the harness `root` rather than the worker directory.
- `services` is replaced with exactly one entry per requested gatekeeper: `{ binding: GATEKEEPER_, service: , entrypoint: "GatekeeperVendor" }`. The checked-in config declares none, so `buildGatekeeperVendorMap()` discovers exactly the vendors the suite asked for and the observer-config prompt has no surprise rows.
- `vars.ADMINS` is set to `[ADMIN_USERNAME]`. No `CF_ACCESS_AUD` is set, so `/api` takes the unauthenticated path and password signup is available.
- `worker_loaders` is deleted. Gadget code is never executed here — a gatekeeper is in observer scope purely by having a `vendorId` — so the Worker Loader is not required to start.
### `Harness`
The wrangler harness itself.
Base URL of the running server, e.g. `http://127.0.0.1:1234`.
Dispatches a request to a named worker's own HTTP entrypoint. The host is never resolved — the request goes straight to that worker — so no `routes` config is needed, but the path must still match what the worker expects. Typed as the harness's own dispatch signature, because this package sees both Node and Workers global types.
## Driving the API over Cap'n Web
`src/rpc-client.ts` speaks the real WebSocket API.
```ts
export function connect(baseUrl: URL): RpcStub {
const wsUrl = new URL("/api", baseUrl);
wsUrl.protocol = wsUrl.protocol === "https:" ? "wss:" : "ws:";
return newWebSocketRpcSession(wsUrl.toString());
}
```
| Helper | Purpose |
|---|---|
| `connect(baseUrl)` | Open an `RpcStub` session against `/api`. |
| `stubFor(target)` | Mint a callback stub. Always use this instead of importing `RpcStub` directly. |
| `signUp(api, username)` | `createAccount` + `authenticate`, returning `RpcStub`. |
| `nextUsernames(...prefixes)` | Fresh per-test usernames: `nextUsernames("alice", "bob")` → `["alice7", "bob7"]`. |
| `listConnectedAccounts(api)` | Drives `subscribeConnectedAccounts()` to its `ready()` call and returns `ConnectedAccount[]`. |
| `accountLabel(account)` | `uniqueName || displayName || \`account ${id}\`` — mirrors the overseer's `#describeObserverFailures` precedence. |
| `waitFor(what, attempt, timeoutMs = 30_000)` | Polls `attempt` every 25 ms until it returns non-null; throws on timeout. |
| `ObserverConfigRecorder` | Records every `configure()` call and answers from a scripted queue. |
| `MAX_OBSERVER_PROMPTS` | `2` — the initial prompt plus at most one re-prompt (`MAX_CONFIG_REPROMPTS` in the overseer is 1). The product does not export it. |
Passwords are not hashed the way the frontend does. The server stores and compares the submitted bytes verbatim and never re-derives them, so the tests substitute a deterministic `sha256("integration-test:" + username)` rather than paying argon2id's 64 MiB per call.
`ObserverConfigRecorder.alwaysChoose(accountId, times)` takes an explicit `times` on purpose: `configure()` throws once the response queue is empty, so an unexpected extra prompt fails the test instead of being silently answered.
## Findings that shape the design
### Fake timers cannot work here
`vi.useFakeTimers()` patches the test process's clock. The code under test reads workerd's clock, out of process, so a faked clock is invisible to it. `isTokenExpired()`'s 30-second skew lives in `gatekeeper-shared` and is evaluated inside the Worker.
Fake timers *do* work for in-isolate unit tests under `vitest-pool-workers`, where the test runs inside the same isolate.
### A fixture gatekeeper, not a real one, for the overseer's own logic
The overseer cases need a gatekeeper that refuses an observer on command. Every shipping public gatekeeper can do that only at a cost that would dominate the test:
- **OAuth gatekeepers** need a whole vendor auth surface mocked before an account exists at all.
- **The Context Library** only refuses once an observation has been *recorded*, which takes a gadget read session (so a Worker Loader), a slash-command invocation, or an AI-chat catalog snapshot. It is also a singleton, so it can never produce the two simultaneously-failing bindings one of these cases needs.
Adding a test hook to those workers was considered and rejected: a "mark observed" hook would stub the very state the tracker maintains, making the test circular.
So `fixtures/gatekeeper-test/` is a real Worker speaking the real protocol, whose verification outcome the tests set over an HTTP control route.
The fixture is scoped to overseer logic, not a long-term substitute for per-vendor coverage. Testing actual gatekeepers is the expected trajectory — which is why the harness takes a list of gatekeepers and the interceptor takes pluggable handler modules. A future `gatekeeper-google` suite is "add `google-handlers.ts`, point the harness at the package", with production code unmodified.
### Storage isolation is by convention, because the alternative is worse
`server.reset()` exists, and measuring it settles the question: **~3 s per call**, which is more than an entire suite run. It also restarts the server — `server.url` becomes undefined and every open WebSocket RPC session dies with `WebSocket connection failed`. It is not a storage wipe you can use between tests; it is a teardown.
Storage therefore persists for the harness's lifetime and **no test may assume a clean slate**. Tests stay independent by taking fresh identities:
- `nextUsernames()` from the toolkit
- per-test resource URLs
- account labels allocated by the connect/provision helper rather than chosen by the caller
One corollary that is easy to get wrong: the "nothing escaped to the internet" assertion belongs in `afterAll`, not `afterEach`. With `it.concurrent`, an `afterEach` fires while siblings are still running, so it would inspect and clear state they are still using — and could discard an escape a sibling was about to be blamed for.
### wrangler and workerd versions are coupled
The public repo pins `workerd` through a root `overrides` entry, which collapses every transitive request to one version. A newer `wrangler` brings a newer `miniflare` that demands a newer `workerd` than the override yields, and the harness then fails to boot:
```text
The Workers runtime failed to start ... requires compatibility date "2026-07-08",
but the newest date supported by this server binary is "2026-06-30".
```
The public package therefore pins `wrangler` to `~4.104.0` — the release whose bundled `workerd` matches the override. Bumping it means bumping the override in step.
### A consumer in another repo can end up with two copies of capnweb
A consumer repo installs its own workspace *and* the `public/` submodule's, as two separate pnpm stores. `capnweb` then resolves to two different copies: the toolkit's `rpc-client` gets the submodule's, while anything importing `capnweb` from one of the consumer's own packages gets the other. A stub is only serialisable by the instance that owns the session, so mixing them fails:
```text
TypeError: Cannot serialize value: [object RpcStub]
```
The trap is that a dev machine where a single `pnpm install` deduped both will not show this. It first appeared in CI, which runs `pnpm install` and `pnpm --dir public install` separately; reproduce locally by doing the same.
The toolkit therefore owns the capnweb boundary: mint callback stubs with `stubFor()` from `rpc-client`, never with an imported `RpcStub`. Importing `RpcStub` as a *type* is fine. This is enforced structurally in this repo — `.oxlintrc.json` restricts `capnweb` value imports within this package to `rpc-client.ts` (`allowTypeImports` leaves type imports alone). A consumer repo without a linter should treat the rule as a convention its test files follow via `stubFor()`.
### Worker entry modules may export only classes and the default handler
workerd treats every named export of the entry module as an entrypoint. Exporting a plain string constant from the fixture produced:
```text
Incorrect type for map entry 'THING_URL_PATTERN': the provided value is not of
type 'function or ExportedHandler'.
```
Type-only exports are fine (they erase). Anything else has to stay module-private.
## Network interception
`src/network-interceptor.ts` is mechanism only. `createTestHarness` routes a Worker's outbound `fetch()` back through the Node process, so patching `globalThis.fetch` is enough — no interception library is needed.
```ts
export type Handler =
(url: URL, method: string, headers: Headers) => Response | null | Promise;
```
A handler answers one request or returns `null` to decline and let the next handler try. Handlers may be async, which is load-bearing rather than a convenience: a handler sometimes has to wait for the test to say what to answer with, because the thing that identifies the request only comes into existence once the Worker has started making it.
Dispatch rules:
1. Requests to `localhost`, `127.0.0.1`, or `[::1]` pass straight through to the real `fetch` — that is the harness's own loopback traffic.
2. Otherwise a `Request` is constructed to normalise method and headers, and the method is upper-cased (`Request` normalises only the methods the fetch spec lists, so `patch` would otherwise reach handlers lowercased). Constructing the `Request` can transfer the body's stream, which is why it happens after the loopback return.
3. Handlers are tried in order; the first non-null `Response` wins.
4. An unmatched request is recorded in `#unmockedCalls` and throws `Unmocked outbound request: `, so an unmocked call fails the test instead of silently reaching the internet.
| Method | Behavior |
|---|---|
| `install()` | Swap in the patched `globalThis.fetch`. Idempotent. |
| `uninstall()` | Restore the real `fetch`. |
| `getUnmockedCalls()` | Copy of the URLs that were neither handled nor local. |
| `takeUnmockedCalls(substring)` | Remove and return matching entries, for the one test that provokes an unmocked request deliberately. Taking just its own entry rather than resetting means a concurrently running sibling's escape is still caught. |
| `reset()` | Clear the recorded unmocked calls. |
What a given vendor's endpoints return lives in a handler module passed to the constructor, so a suite for another gatekeeper is a new handler module rather than a fork of this file.
## The fixture gatekeeper
`fixtures/gatekeeper-test/src/test-gatekeeper.ts` is a real Worker implementing the real gatekeeper protocol against vendor host `gadgets-test.example`.
```mermaid
classDiagram
class GatekeeperVendor {
+describe() VendorDescription
+createAccount() Fetcher~GatekeeperUser~
+getSupportedResources() SupportedResource[]
+getTypeScriptTypes() string
+connectAccount() throws
}
class TestAccount {
+describe() AccountDescription
+getSingletonGatekeeperClass()
+getSupportedResources()
+getGatekeeperClassFor(url)
}
class TestControl {
+setVerifyOutcome(label, outcome)
+getVerifyOutcome(label) VerifyOutcome
+recordAmbientVerification(label)
+getAmbientVerificationCount(label) number
}
class TestGatekeeper {
props: label, resourceUrl, ambient?
}
GatekeeperVendor --> TestAccount : createAccount()
TestAccount --> TestGatekeeper : getGatekeeperClassFor()
TestGatekeeper ..> TestControl : reads verify outcome
```
Design points that matter when reading the tests:
- `autoProvisionsAccount: true` in `describe()`. Accounts are minted on request with no auth flow, which is what keeps these tests about the overseer rather than about somebody's OAuth dance. `connectAccount()` is required by the interface but unreachable, and throws.
- `createAccount()` mints a distinct account per call — a random `test-<12 hex>@gadgets-test.example` label — so two users, or two concurrent tests, never share one.
- `AccountDescription.uniqueName` is that label. It is what the overseer names in a verification-failure message, and the same string the Workshop shows the user, so control state is keyed on it: a test that read a label off `description.uniqueName` can aim an outcome at it without learning any internal id.
- `TestControl` is a Durable Object storing `outcome: