# Troubleshooting

> Known failure modes and their fixes: `OPEN_GADGET_ERROR_CODES` and observer binding failures, GitHub App versus OAuth App scope errors, missing generated modules on a clean checkout, gadget lockdown from `prohibitAllSharing`, free-tier and balance blocks, missing `WORKERS_AI` binding in gateway mode, no-op error reporting without its bindings, and RPC stub leaks or `useState` misuse.

- Repository: cloudflare/cloudflare-os
- GitHub: https://github.com/cloudflare/cloudflare-os
- Human docs: https://grok-wiki.com/public/docs/cloudflare-cloudflare-os-838773bb92dd
- Complete Markdown: https://grok-wiki.com/public/docs/cloudflare-cloudflare-os-838773bb92dd/llms-full.txt

## Source Files

- `packages/workshop-shared/src/api.ts`
- `packages/gatekeeper-github/README.md`
- `docs/ai-gateway-billing.md`
- `docs/public-server.md`
- `packages/backend-utils/src/error-reporting.ts`
- `packages/workshop-backend/src/client-errors.ts`
- `AGENTS.md`

---

---
title: "Troubleshooting"
description: "Known failure modes and their fixes: `OPEN_GADGET_ERROR_CODES` and observer binding failures, GitHub App versus OAuth App scope errors, missing generated modules on a clean checkout, gadget lockdown from `prohibitAllSharing`, free-tier and balance blocks, missing `WORKERS_AI` binding in gateway mode, no-op error reporting without its bindings, and RPC stub leaks or `useState` misuse."
---

Most failures in this stack come from a missing optional binding, an OAuth app registered in the wrong GitHub product, or an environment variable pair that must be set together. The optional-binding cases are deliberate no-ops rather than errors: `reportIssue()` returns immediately when `ERROR_REPORTER` is absent, and `handleClientErrorRequest()` returns `204` when either `FRONTEND_ERROR_REPORTER` or `FRONTEND_ERROR_RATE_LIMITER` is unbound. Silence is the expected symptom, not a bug.

## Sign-in and gatekeeper OAuth

### `Resource not accessible by integration`

The deployment is configured with a **GitHub App** (client id beginning `Iv…`) rather than a GitHub **OAuth App**. GitHub Apps ignore the OAuth `scope` parameter entirely, so the `user:email` read that sign-in depends on is forbidden and sign-in is rejected.

<Warning>
Use a GitHub **OAuth App**, not a GitHub **App**. Only OAuth Apps honor the `scope` parameter, which is what makes minimal-scope login (`read:user user:email`) and full-scope connect (`repo read:user user:email`) behave differently.
</Warning>

Two fixes:

<Tabs>
<Tab title="Switch to an OAuth App (recommended)">
Register at GitHub Settings → Developer settings → **OAuth Apps** → **New OAuth App**, with authorization callback URL `http://localhost:8787/gatekeeper/github/oauth` (replace the host with `PUBLIC_BASE_URL` when not local). No extra permission setup is needed — the `user:email` scope is requested automatically.
</Tab>
<Tab title="Keep the GitHub App">
Grant the App the **Email addresses** account permission: App settings → **Permissions & events** → **Account permissions** → **Email addresses → Read-only** → save. Existing users must then re-run the sign-in flow to approve the added permission. Login still cannot be minimal-scope, because the App's fixed permissions apply to every authorization.
</Tab>
</Tabs>

### `redirect_uri_mismatch`

The callback URL registered on the OAuth app does not match what the gatekeeper sends. The contract is `${PUBLIC_BASE_URL}/gatekeeper/<name>/oauth`, lowercase and exact:

| Gatekeeper | Redirect URI |
| --- | --- |
| GitHub | `${PUBLIC_BASE_URL}/gatekeeper/github/oauth` |
| Google | `${PUBLIC_BASE_URL}/gatekeeper/google/oauth` |
| Cloudflare | `${PUBLIC_BASE_URL}/gatekeeper/cloudflare/oauth` |

For local development that is `http://localhost:8787/gatekeeper/github/oauth` — no trailing slash, `http` not `https`.

### `bad_verification_code`

The authorization code expired or was already redeemed. Return to the app and start the connect flow again.

### "Not configured" page during authorization

The gatekeeper Worker has no `CLIENT_ID` / `CLIENT_SECRET`. For the GitHub connector, create `packages/gatekeeper-github/.env` with both values and restart the dev server:

```bash title="packages/gatekeeper-github/.env"
CLIENT_ID=your-client-id-here
CLIENT_SECRET=your-client-secret-here
```

The file is gitignored and must never be committed. In dev, gatekeeper credentials are seeded from shell variables (`GITHUB_*`, `GOOGLE_*`, `CLOUDFLARE_OAUTH_*`) by `run-dev-server.js`.

### "Continue with …" button missing on the login page

`AUTH_GATEKEEPERS` is the allowlist that decides which connected gatekeepers may be used to sign in. Without the vendor listed, `PublicApi.startGatekeeperLogin(vendorId)` throws — the vendor must be both auth-capable and allowlisted (see `ServerConfig.authVendors`).

```
AUTH_GATEKEEPERS=cloudflare,google,github
```

<Note>
`DISABLE_PASSWORD_AUTH=true` is ignored unless `AUTH_GATEKEEPERS` is non-empty. That guard exists specifically to avoid locking every user out of a deployment with no working sign-in path.
</Note>

Because the primary account key is always the user's **verified email**, signing in through any allowlisted gatekeeper that yields the same verified email lands on the same account. A user who "lost their account" after switching sign-in providers usually has a different verified email on the second provider.

## Free-tier and balance blocks

These only occur when `ENABLE_CLOUDFLARE_LIMITS=true`. Unset, usage is unlimited, which is the self-hosted default. Before each user-initiated agent turn the overseer calls `checkUsageAndBalance`, which resolves to one of four outcomes:

| User state | Outcome | Routing |
| --- | --- | --- |
| Connected, balance ≥ `$2` | Allowed | The user's own account; bills their Cloudflare credits; daily free-tier counter untouched |
| Within free tier (incl. connected with balance below `$2`, or `$0`) | Allowed | The platform's configured AI Gateway |
| Free tier exhausted, no Cloudflare account connected | Blocked | Prompt to connect |
| Free tier exhausted, connected but balance below `$2` | Blocked | Prompt to add credits |

<Steps>
<Step title="Confirm which block you hit">
"Connect" prompt means no Cloudflare gatekeeper connection exists. "Add credits" means the connection exists but the live balance is under the minimum.
</Step>
<Step title="Connect Cloudflare">
Sign in with Cloudflare, or use the "Connect Cloudflare" button, which runs the normal gatekeeper connect flow (`AuthenticatedApi.connectAccount("cloudflare")`). Billing reads a usable token from that gatekeeper's connection via `getUsableAccessToken()`; the `UserDurableObject` stores only the selected account id, a cached balance, and the daily counter — never tokens.
</Step>
<Step title="Top up">
Add credits in the [Cloudflare dashboard](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway). The platform never holds money.
</Step>
<Step title="Wait out the balance cache">
The balance is read live from `/ai-gateway-billing/credit_balance` and **cached for 5 minutes**. A top-up that appears not to take effect is usually a stale cache entry.
</Step>
</Steps>

### Tuning the thresholds

```
DAILY_LLM_CALL_LIMIT=100        # free-tier LLM calls per user per UTC day
MINIMUM_CLOUDFLARE_BALANCE=2    # min connected-account balance (USD) to proceed via BYOK
```

The daily counter lives on each `UserDurableObject` (`consumeDailyLlmCall` / `checkDailyLlmCount`) and resets per UTC day — there is no separate binding to inspect or clear.

### Account selection stalls

The account to bill is auto-selected when the OAuth grant sees exactly one account. With several accounts, the user must pick one; that prompt is rendered by `components/billing/AccountSelectionModal`. Billing is account-level (Unified Billing) and inference routes through the account's auto-created "default" AI Gateway.

## AI Gateway configuration failures

### Missing `CF_AI_GATEWAY_ACCOUNT_ID` / `CF_AI_GATEWAY_API_TOKEN`

Gateway mode always requires both. All inference goes over HTTPS with tokens, and the API token needs **AI Gateway Run and Read** permissions — Read is what lets gadgets retrieve each log's cost for user-visible accounting. Omitting Read produces working inference with no cost reporting.

```
CF_AI_GATEWAY=your-gateway
CF_AI_GATEWAY_PROVIDERS=anthropic,openai,google
CF_AI_GATEWAY_ACCOUNT_ID=...
CF_AI_GATEWAY_API_TOKEN=...
```

The Gateway may live in the Worker's own account or a different one.

### Missing `WORKERS_AI` binding in local gateway mode

When any `CF_AI_GATEWAY*` variable is set in local development, start the server with the binding flag so the `webFetch` tool's document-to-Markdown conversion still has a `WORKERS_AI` binding:

```bash
pnpm run dev-server -- --use-workers-ai-binding
```

<Info>
Inference itself no longer uses the binding — it goes over HTTPS with the tokens above. Only the document-conversion path in `webFetch` still needs `WORKERS_AI`, which is why the flag is opt-in rather than implied by `CF_AI_GATEWAY`.
</Info>

### Workers AI routed unexpectedly

Workers AI uses `CF_AI_GATEWAY` as its Gateway ID by default. Two overrides change that:

| Variable | Effect |
| --- | --- |
| `CF_AI_GATEWAY_WAI` | Route Workers AI through a different Gateway in the same account |
| `CF_AI_GATEWAY_WAI_DIRECT=true` | Bypass gateways and call the Workers AI REST endpoint directly, using the same account/token pair |

`CF_AI_GATEWAY_WAI_DIRECT=true` produces **no gateway cost logs**. If per-call cost disappeared from the UI, check this flag first.

### Cloudflare OAuth endpoints look wrong

The Cloudflare dashboard OAuth endpoints and scopes are hardcoded in `packages/gatekeeper-cloudflare/src/oauth.ts` and are not configurable:

```
auth:   https://dash.cloudflare.com/oauth2/auth
token:  https://dash.cloudflare.com/oauth2/token
scopes: offline_access aig.read aig.run aig.write user-details.read account-settings.read
```

## Error reporting silently does nothing

Both reporting paths degrade to no-ops rather than failing, by design.

```text
 backend capture site                    browser report
        │                                      │
 reportIssue(failureSite, caught)      POST /api/... (JSON)
        │                                      │
        ├─ env.ERROR_REPORTER absent?          ├─ FRONTEND_ERROR_REPORTER absent?  ──┐
        │     └─> return (no-op)               ├─ FRONTEND_ERROR_RATE_LIMITER absent? ┤─> 204
        │                                      │                                     │
        └─ waitUntil(report(event))            └─ limiter.limit({key}) not success ───┘
              └─ .catch -> logger.debug                     │
                                                            └─> reporter.report(event)
```

<AccordionGroup>
<Accordion title="Backend: no events arrive from reportIssue()">
`reportIssue()` returns immediately when the optional `ERROR_REPORTER` service binding is absent — expected in local dev and in deployments without an Issue destination. When the binding *is* present, dispatch failures are logged at `debug` under `error_report.dispatch.failed`, and setup failures under `error_report.setup.failed`, both from the `backend-utils.error-reporting` component. Raise log verbosity to see them; reporting never disturbs the caller.
</Accordion>
<Accordion title="Frontend: /client-errors always returns 204">
`handleClientErrorRequest()` returns `204` with no body when either `FRONTEND_ERROR_REPORTER` or `FRONTEND_ERROR_RATE_LIMITER` is missing, and also when the rate limiter denies the request or the limiter call throws (`frontend_error_report.rate_limit.failed`). A `204` is therefore not evidence that a report was delivered.
</Accordion>
<Accordion title="Frontend: non-204 rejections">
| Status | Cause |
| --- | --- |
| `405` | Method is not `POST` (response carries `allow: POST`) |
| `403` | `origin` header does not equal the request URL origin, or CF Access JWT invalid, or the Access JWT specified no user identity |
| `415` | `content-type` is not `application/json` |
| `413` | Body exceeds `MAX_BODY_BYTES` (128 KiB), by declared `content-length` or measured stream length |
| `400` | Body is not parseable JSON, or `normalizeFrontendErrorReport()` rejected the shape |

When `CF_ACCESS_AUD` is set, the rate-limit key comes from the verified Access JWT (`accessRateLimitKey`); otherwise it falls back to `cf-connecting-ip`, or the literal `unknown`.
</Accordion>
<Accordion title="Events look incomplete or truncated">
Both event builders bound their inputs and set `truncated: true` when anything was cut. Backend `attributes` drop non-scalar values, drop duplicate keys, cap at `MAX_ATTRIBUTE_KEYS`, and clamp strings to `MAX_STRING_CHARS`. `http.kind` is normalized to exactly `"server"` or `"client"` because callers cross a JS trust boundary where the union is not enforced at runtime. A missing field usually means it was dropped by normalization, not lost in transit.
</Accordion>
</AccordionGroup>

## RPC session and stub problems

The client/server API is a Cap'n Web RPC interface over a single WebSocket opened at startup and kept open for the session lifetime, reconnecting as needed. Two consequences show up as bugs.

### Abandoned sign-in attempts

`PublicApi.startGatekeeperLogin(vendorId)` returns `{ url, attempt }` where `attempt` is an `RpcStub<LoginAttempt>`. Holding that stub *is* the capability to receive the resulting session token.

<ParamField body="url" type="string" required>
Opened by the client in a new tab; the gatekeeper's OAuth popup self-closes on completion.
</ParamField>

<ParamField body="attempt" type="RpcStub<LoginAttempt>" required>
`attempt.wait()` resolves with a session token to store and pass to `authenticate()`, and is safe to call immediately after `startGatekeeperLogin()`. It rejects if the attempt fails or is abandoned.
</ParamField>

Dispose `attempt` when the user closes the popup — disposal is what cancels the server-side wait. Failing to dispose leaks the stub and leaves a pending server-side wait; disposing while still expecting a token turns into a `wait()` rejection.

### Authentication paths

| Call | Use when |
| --- | --- |
| `authenticate(token)` | Normal token flow; token typically from `localStorage` |
| `authenticateFromCfAccess()` | The server sits behind Cloudflare Access and the browser already authenticated with Access |
| `login(username, passwordHash)` | Username/password; returns `null` for no such user or wrong password |
| `createAccount(username, displayName, passwordHash)` | Returns `null` if the username already exists; other errors throw |

`login()` and `createAccount()` may be disabled when the server uses SSO — a `login()` path that stops working after enabling SSO is expected, not broken.

`passwordHash` must be derived exactly as specified, or login fails against an account created with different parameters:

```js
argon2id({
  password,
  salt: SERVICE_SALT + encode(username, 'utf8'),
  parallelism: 1,
  iterations: 3,
  memorySize: 64MiB,
  hashLength: 32,
});
```

`SERVICE_SALT` is the fixed 16-byte constant exported from `packages/workshop-shared/src/api.ts`. The server never sees the plaintext password; it hashes the submitted `passwordHash` again server-side.

### Stale connected-account credentials

`subscribeConnectedAccounts()` delivers each account through `add(id, description, vendor, supportedResources, credentialsValid, vendorId)`. When `credentialsValid` is `false`, the account's credentials are known to be expired: the UI must call `reconnectAccount()` if the user selects it. Treating an expired account as usable produces downstream gatekeeper failures rather than a clear reconnect prompt. `ready()` fires after `add()` has been called for every account known so far — reading the list before `ready()` yields a partial set.

### Blueprint fetches that return `null`

`getBlueprint(id)` returns `null` when the blueprint does not exist and requires no authentication: knowing the ID is sufficient, because a blueprint is "just data". `downloadBlueprint(id)` streams a `.gadget` archive containing only `BlueprintMetadata` plus the current code snapshot — not the full KV record. An import that appears to lose state is usually this omission, not a corrupted archive.

### Gadget iframe cannot reach the network

By design. Gadgets run in a sandboxed iframe with no ability to talk to the outside world except `postMessage()` to the parent frame; RPC to the Workshop is bridged over those exchanges, and the Workshop hands the gadget a stub pointing at its own server-side Durable Object interface. Direct `fetch()` from gadget UI code is not a supported path.

## Local development environment

<Steps>
<Step title="Create a root .dev.vars">
Set required variables in a root `.dev.vars` file — gitignored, one `KEY=VALUE` per line. `pnpm run dev-server` loads it automatically. Variables placed elsewhere are not picked up.
</Step>
<Step title="Set PUBLIC_BASE_URL to the router port">
Locally that is `PUBLIC_BASE_URL=http://localhost:8787`. Every gatekeeper redirect URI is derived from it, so a mismatch here surfaces as `redirect_uri_mismatch`.
</Step>
<Step title="Add the Workers AI flag if using gateway mode">
`pnpm run dev-server -- --use-workers-ai-binding`.
</Step>
<Step title="Verify a gatekeeper end to end">
Open a gadget → **Connections** tab → **+ New Connection** → choose a GitHub resource type (repository, issue, or pull request) → connect the account. GitHub's authorization page opens in a new tab, the tab closes on grant, and the picker lets you select the exact resource. The gadget then has access only to the selected resource scope. Connected accounts can be added and removed from settings via the account menu in the upper right.
</Step>
</Steps>

A minimal working `.dev.vars` for the public multi-user posture:

```bash title=".dev.vars"
ENABLE_CLOUDFLARE_LIMITS=true
PUBLIC_BASE_URL=http://localhost:8787
AUTH_GATEKEEPERS=cloudflare,google,github

GITHUB_CLIENT_ID=...
GITHUB_CLIENT_SECRET=...
GOOGLE_CLIENT_ID=...
GOOGLE_CLIENT_SECRET=...
CLOUDFLARE_OAUTH_CLIENT_ID=...
CLOUDFLARE_OAUTH_CLIENT_SECRET=...

CF_AI_GATEWAY=your-gateway
CF_AI_GATEWAY_PROVIDERS=anthropic,openai,google
CF_AI_GATEWAY_ACCOUNT_ID=...
CF_AI_GATEWAY_API_TOKEN=...
```

## Where billing code lives

When tracing a limits or balance bug, server code is under `packages/workshop-backend/src/ai-gateway-billing/`:

:::files
```
ai-gateway-billing/
├── config.ts                     # ENABLE_CLOUDFLARE_LIMITS / minimum-balance readers
├── limits/
│   ├── config.ts                 # daily-limit + calendar-day helpers + DailyQuotaResult
│   └── usage-checker.ts          # checkUsageAndBalance / getUsageInfo
└── cloudflare/
    ├── account-service.ts        # CF REST: accounts / balance
    └── connection-service.ts     # token, account selection, balance cache, BYOK routing
```
:::

Client side, `ServerConfigContext` exposes `cloudflareLimitsEnabled`, and `components/billing/` (`UsageSettings`, `OutOfCreditsModal`, `AccountSelectionModal`) renders the usage, top-up, and account-selection UI.

## Related pages

<CardGroup cols={2}>
<Card title="Configure gatekeeper credentials" href="/configure-gatekeeper-credentials">
The `${PUBLIC_BASE_URL}/gatekeeper/<name>/oauth` contract, per-connector `CLIENT_ID`/`CLIENT_SECRET`, and dev seeding from shell variables.
</Card>
<Card title="Configure sign-in and AI Gateway billing" href="/configure-signin-and-billing">
`AUTH_GATEKEEPERS`, `DISABLE_PASSWORD_AUTH`, email-keyed identity, and the `ENABLE_CLOUDFLARE_LIMITS` free-tier plus top-up flow.
</Card>
<Card title="Environment variables" href="/environment-variables">
Every backend variable and its default, including the `CF_AI_GATEWAY*` family and the frontend error-reporting switch.
</Card>
<Card title="RPC API reference" href="/rpc-api-reference">
`PublicApi`, `LoginAttempt`, `AuthenticatedApi`, and the stub-disposal and promise-pipelining constraints.
</Card>
<Card title="Local development" href="/local-development">
`.dev.vars` loading, the two-terminal workflow, and the `--use-workers-ai-binding` flag.
</Card>
<Card title="Routing and worker bindings" href="/routing-and-bindings">
How `/gatekeeper/<name>/*` routes are derived and which backend bindings are optional.
</Card>
</CardGroup>