# Inference router

> SandInferenceProvider ids cursor, claude-code, codex, openrouter; default cursor; local transcript store schemaVersion 2; usage schemaVersion 1.

- Repository: sashimikun/grok-bot-0.18-reconstructed
- GitHub: https://github.com/sashimikun/grok-bot-0.18-reconstructed
- Human docs: https://grok-wiki.com/public/docs/sashimikun-grok-bot-0-18-reconstructed-c774cc9a5c15
- Complete Markdown: https://grok-wiki.com/public/docs/sashimikun-grok-bot-0-18-reconstructed-c774cc9a5c15/llms-full.txt

## Source Files

- `source/shared/inference-router.ts`
- `source/node-agent-coordinator/inference-router.ts`
- `source/host/extensions/inference/provider-session.ts`
- `source/electron-main/main-edge.ts`
- `frontend/src/recovered/features/settings/overlay/router.ts`
- `tests/inference-router-transcript.test.mjs`

---

---
title: "Inference router"
description: "SandInferenceProvider ids cursor, claude-code, codex, openrouter; default cursor; local transcript store schemaVersion 2; usage schemaVersion 1."
---

The inference router is the shared `SandInferenceProvider` selection that decides whether a turn uses the Cursor-hosted agent path or a reconstructed direct provider. The four ids are `cursor`, `claude-code`, `codex`, and `openrouter`. Missing or invalid values resolve to `cursor`. The packaged Settings → Router page writes the selection through `window.desktop.agent.setInferenceRouter`, persists `inferenceProvider` on `settings.json` (`version: 1`), records usage as `inferenceRouterUsage` with `schemaVersion: 1`, and stores non-Cursor conversation tails in `inference-router-transcript.json` with `schemaVersion: 2`.

<Note>
Cursor remains the default and is not intercepted. Direct providers (`claude-code`, `codex`, `openrouter`) own `sendPrompt`, merge local transcript tails, and execute Grok Bot plugin tools through `listRoutedMcpTools` / `executeRoutedMcpTool`.
</Note>

## Provider ids

| Id | Settings label | Auth surface | Turn path | Plugin tools |
| --- | --- | --- | --- | --- |
| `cursor` | Cursor | Existing Grok Bot / Cursor session | Native host inference session | Native Grok Bot tools and plugins |
| `claude-code` | Claude Code | Local Claude Code CLI login | Claude Agent SDK `query` | HTTP MCP bridge `grok_bot_plugins` |
| `codex` | Codex | Private `~/.codex/auth.json` ChatGPT tokens | Direct Responses at `https://chatgpt.com/backend-api/codex/responses` | Direct tool-execution loop |
| `openrouter` | OpenRouter | `OPENROUTER_API_KEY` (env or `box-secrets.json`) | OpenAI-compatible `https://openrouter.ai/api/v1` | Direct tool-execution loop |

Shared type:

```ts title="source/shared/inference-router.ts"
export const SAND_INFERENCE_PROVIDERS = ["cursor", "claude-code", "codex", "openrouter"] as const;
export type SandInferenceProvider = (typeof SAND_INFERENCE_PROVIDERS)[number];
```

`isSandInferenceProvider` is the only accepted setter check. `setInferenceRouter` throws `Unknown inference provider.` for any other string.

## Runtime ownership

```mermaid
flowchart TB
  subgraph ui [Renderer]
    RouterPanel["Settings → Router"]
    UsagePanel["Settings → Usage & Billing"]
    DesktopAgent["window.desktop.agent"]
  end
  subgraph main [Electron main]
    GetSet["getInferenceRouter / setInferenceRouter"]
    SettingsStore["SandSettingsStore settings.json"]
    LocalStatus["getLocalInferenceCliStatus"]
  end
  subgraph coordinator [Node agent coordinator]
    Dispatch["createCoordinatorInferenceRouter.dispatch"]
    Transcript["inference-router-transcript.json"]
    RemoteGW["gatewayClient.dispatchCommand"]
  end
  subgraph host [Host]
    TurnShell["turn-run-shell / createHostInference"]
    CursorPath["createCursorSandInference"]
    ProviderSession["runRoutedProviderText"]
  end
  RouterPanel --> DesktopAgent
  UsagePanel --> DesktopAgent
  DesktopAgent --> GetSet
  GetSet --> SettingsStore
  GetSet --> LocalStatus
  GetSet -->|"syncHostSettingsToBox({ inferenceProvider })"| host
  Dispatch -->|"provider === cursor"| RemoteGW
  Dispatch -->|"provider !== cursor and sendPrompt"| ProviderSession
  Dispatch --> Transcript
  TurnShell -->|"inferenceProvider === cursor"| CursorPath
  TurnShell -->|"else"| ProviderSession
```

Three layers read the same `settings.json` field:

- Electron main for Settings RPC and local CLI status.
- Host `SettingsService` / `SandSettingsStore` for `createHostInference` and `turn-run-shell`.
- Coordinator `createCoordinatorInferenceRouter` for intercepting renderer `sendPrompt` before the remote gateway.

Cursor turns stay on the original host inference client. Direct providers replace that session with `createProviderPromptSession` / `runRoutedProviderText`.

## Select and persist a provider

Packaged UI does **not** use the frontend design-workspace key `settings.router-provider.v1`. Packaging tests require the shipped renderer patch to call desktop RPC instead.

<ParamField body="provider" type="SandInferenceProvider" required>
One of `cursor`, `claude-code`, `codex`, `openrouter`. Default when unset: `cursor`.
</ParamField>

:::endpoint GET getInferenceRouter
Read the current route, usage snapshot, and local CLI status.

**Channel:** `window.desktop.agent.getInferenceRouter()` → main-edge `getInferenceRouter`

**Returns:** `{ provider, usage, local }`

- `provider` — stored `inferenceProvider`, coerced with `isSandInferenceProvider`, else `cursor`
- `usage` — host `inferenceRouterUsage` when reachable, else the local settings copy
- `local` — `{ codex, "claude-code" }` installation/auth probes

`openrouter` status is not in `local`; the Router panel checks the secrets list for `OPENROUTER_API_KEY`.
:::

:::endpoint POST setInferenceRouter
Persist a provider id and mirror it into host settings.

**Channel:** `window.desktop.agent.setInferenceRouter(provider)` → main-edge `setInferenceRouter` with `{ provider }`

**Behavior:**

1. Reject unknown ids with `Unknown inference provider.`
2. Write `inferenceProvider` through `SandSettingsStore.setInferenceProvider`.
3. Call `syncHostSettingsToBox({ inferenceProvider })`. Host sync failure is swallowed; the local write still stands.
4. Return `{ provider, usage, local }` using host usage when the sync response includes it.

The patched Router panel then dispatches `sand-router-provider-changed` with that payload.
:::

<RequestExample>
```js title="Renderer"
await window.desktop.agent.setInferenceRouter("codex");
```
</RequestExample>

<ResponseExample>
```json title="getInferenceRouter / setInferenceRouter"
{
  "provider": "codex",
  "usage": {
    "schemaVersion": 1,
    "providers": {
      "cursor": { "requests": 0, "inputTokens": 0, "outputTokens": 0, "cacheReadTokens": 0, "cacheWriteTokens": 0, "lastUsedAt": null },
      "claude-code": { "requests": 0, "inputTokens": 0, "outputTokens": 0, "cacheReadTokens": 0, "cacheWriteTokens": 0, "lastUsedAt": null },
      "codex": { "requests": 3, "inputTokens": 1200, "outputTokens": 400, "cacheReadTokens": 0, "cacheWriteTokens": 0, "lastUsedAt": "2026-08-24T12:00:00.000Z" },
      "openrouter": { "requests": 0, "inputTokens": 0, "outputTokens": 0, "cacheReadTokens": 0, "cacheWriteTokens": 0, "lastUsedAt": null }
    }
  },
  "local": {
    "codex": { "installed": true, "authenticated": true, "executablePath": "/opt/homebrew/bin/codex" },
    "claude-code": { "installed": false, "authenticated": false, "executablePath": null }
  }
}
```
</ResponseExample>

`settings.json` lives under the Sand data root (`SAND_DATA_ROOT`, else `<userData>/sand-data`, else `~/.grokbot` for production `sand`). Writes are atomic (`settings.json.<pid>.tmp` then rename). Host `setHostSettings` only accepts a provider that passes `isSandInferenceProvider`.

The Vite `frontend/` overlay still round-trips `{ schemaVersion: 1, provider }` under `settings.router-provider.v1` for the design workspace. That key is not the packaged persistence path.

## Direct-provider intercept

Coordinator dispatch order:

1. `reactToMessage` — toggle a local `by: "me"` reaction on a stored entry if one exists; emit `transcript` `updated`.
2. Transcript reads (`getAgentTranscriptTail`, `openAgentTail`, `getAgentTranscriptWindow`) when provider is not `cursor` — remote entries plus projected local entries, sliced to `limit` (default `500`).
3. `sendPrompt` when provider is not `cursor` — handle locally and return `{ accepted: true, clientNonce, provider }` immediately. Turns for the same `agentId` are serialized on an in-process queue.
4. Otherwise `{ handled: false }` so the remote gateway runs.

```mermaid
sequenceDiagram
  participant Renderer
  participant Coordinator as createCoordinatorInferenceRouter
  participant Transcript as inference-router-transcript.json
  participant Host as runRoutedProviderText
  participant Gateway as gatewayClient
  Renderer->>Coordinator: sendPrompt({ agentId, prompt, richText, clientNonce })
  alt provider is cursor
    Coordinator->>Gateway: dispatchCommand(sendPrompt)
  else provider is claude-code, codex, or openrouter
    Coordinator->>Transcript: append user t{n}u
    Coordinator->>Renderer: transcript appended + agents thinking pulse
    Note over Coordinator: wait 1200 ms for composing row
    Coordinator->>Host: runRoutedProviderText + onTextDelta
    Host-->>Coordinator: streamed text then final content
    Coordinator->>Transcript: append assistant t{n}s0
    Coordinator-->>Renderer: { accepted, clientNonce, provider }
  end
```

Local turn constraints:

| Rule | Value |
| --- | --- |
| Required args | Non-empty `agentId` and `prompt` |
| Optional args | `richText` (string), `clientNonce` (else a new UUID) |
| Turn ids | Next `t{n}` after max of remote and local `/^t(\d+)(?:u\|s\d+)$/` ids |
| User projection | `{ kind: "message", id: "t{n}u", role: "user", ... }` |
| Assistant projection | `{ kind: "send-message", id: "t{n}s0", message: { type: "text", content } }` |
| Activity pulse | `currentActivity: { kind: "thinking" }` every 250 ms until the turn settles |
| Composing delay | 1200 ms before the first assistant stream so the shipped transcript can show composing |
| Failure | Assistant row `Router error: <message>` |

## Local transcript store

Path: `<dataDir>/inference-router-transcript.json`  
Mode: `0o600` temp file, then rename  
Parser: `parseInferenceRouterTranscriptStore` — any root whose `schemaVersion` is not `2` or whose `agents` is not an object becomes `{ schemaVersion: 2, agents: {} }`

```json title="inference-router-transcript.json"
{
  "schemaVersion": 2,
  "agents": {
    "<agentId>": [
      {
        "provider": "codex",
        "role": "user",
        "content": "@Gmail what's new?",
        "richText": "{\"type\":\"doc\",...}",
        "id": "t1u",
        "clientNonce": "nonce-1",
        "timestampMs": 123,
        "reactions": [{ "emoji": "👍", "by": "me" }]
      },
      {
        "provider": "codex",
        "role": "assistant",
        "content": "...",
        "id": "t1s0",
        "timestampMs": 456
      }
    ]
  }
}
```

<ResponseField name="schemaVersion" type="2">
Required. Any other version is discarded.
</ResponseField>
<ResponseField name="agents" type="Record<agentId, StoredEntry[]>">
Per-agent tails. Each agent is truncated to the last 200 valid entries on parse and on append.
</ResponseField>
<ResponseField name="provider" type='"claude-code" | "codex" | "openrouter"'>
Cursor rows are never stored here.
</ResponseField>
<ResponseField name="richText" type="string">
Optional. Must be a string; object carriers are dropped. Used to keep structured MCP mention docs across reload.
</ResponseField>

Malformed rows are skipped, not migrated. `richText` that is not a string rejects the whole entry.

## Usage store

`inferenceRouterUsage` on `settings.json` is `schemaVersion: 1` and always has all four provider keys.

```ts title="SandInferenceRouterUsageProvider"
{
  requests: number;
  inputTokens: number;
  outputTokens: number;
  cacheReadTokens: number;
  cacheWriteTokens: number;
  lastUsedAt: string | null;
}
```

`SandSettingsStore.recordInferenceUsage` increments `requests` by 1, adds finite non-negative token counts (rounded), and sets `lastUsedAt` to `new Date().toISOString()`. Direct providers record from streamed usage; the Cursor host path records from `extendedUsage` on the wrapped executor.

These counters are local activity records. They are not a provider invoice. The patched Usage & Billing page shows **Tracked activity** for the current route plus any provider with `requests > 0`, and still mounts the original Cursor usage panel when the current route is `cursor`.

## Provider clients

Direct turns share this system prompt:

> You are Grok Bot, a warm, concise desktop assistant. You are running inside Grok Bot, not inside Codex CLI or Claude Code. The tools supplied with this request are Grok Bot's already-connected plugins and accounts.

| Provider | Client | Model default | Credential check | Tool max steps |
| --- | --- | --- | --- | --- |
| `claude-code` | `@anthropic-ai/claude-agent-sdk` `query` | `SAND_CLAUDE_MODEL` if set, else Claude Code default | `resolveClaudeCodeCliPath()` must find a binary | 8 with MCP URL, else 1 |
| `codex` | `streamCodexDirectResponses` | `SAND_CODEX_MODEL`, else `model` in `config.toml`, else `gpt-5.4` | Regular non-symlink `auth.json` with mode bits `0o077 == 0`, `auth_mode === "chatgpt"`, and access/refresh/id/account tokens | 8 with tools, else 1 |
| `openrouter` | `@ai-sdk/openai` `streamText` | `SAND_OPENROUTER_MODEL` or `openai/gpt-5.2` | `OPENROUTER_API_KEY` from env, else `box-secrets.json` | 8 with tools, else 1 |

`claude-code` MCP: coordinator starts `createRoutedMcpBridge` on `127.0.0.1:<ephemeral>/mcp/<secret>` and passes that URL as `mcpServers.grok_bot_plugins`. Allowed tools: `mcp__grok_bot_plugins__*`. `persistSession` is `false`.

`codex` / `openrouter`: coordinator lists tools with `listRoutedMcpTools` and executes with `executeRoutedMcpTool` (`providerIdentifier`, `name`, `toolName`, `args`, `toolCallId`, `agentId`).

Codex HTTP uses `Authorization: Bearer <access_token>` and `ChatGPT-Account-Id`. A `401` refreshes against `https://auth.openai.com/oauth/token` and rewrites `auth.json` atomically at mode `0o600`.

## Local CLI status

`getLocalInferenceCliStatus()` is what Settings shows as Ready / Sign in / Not installed.

| Probe | `installed` | `authenticated` | Path resolution |
| --- | --- | --- | --- |
| `codex` | `auth.json` exists | Usable ChatGPT token file (same privacy rules as the executor) | `CODEX_PATH`, `~/.local/bin/codex`, `~/.codex/bin/codex`, `PATH`, Homebrew paths. The CLI binary is **not** on the request path. |
| `claude-code` | Resolved `claude` binary | `~/.claude/.credentials.json` exists **or** `ANTHROPIC_API_KEY` is non-empty | `CLAUDE_CODE_PATH`, `~/.local/bin/claude`, `~/.claude/local/claude`, `PATH`, Homebrew paths |

`CODEX_HOME` overrides the Codex home used for `auth.json` and `config.toml` (default `~/.codex`).

## Errors the router emits

| Condition | Message |
| --- | --- |
| Unknown `setInferenceRouter` id | `Unknown inference provider.` |
| Direct `sendPrompt` missing `agentId` or `prompt` | `Local inference routing requires an agentId and prompt` |
| OpenRouter key missing | `OpenRouter needs OPENROUTER_API_KEY. Add it in Settings → Router.` |
| Claude Code binary missing | `Claude Code is not installed. Install and sign in to Claude Code, then reopen Grok Bot.` |
| Codex `auth.json` not a private regular file | `Codex login credentials must be a private direct regular file.` |
| Codex not ChatGPT-signed-in | `Codex is not signed in with ChatGPT. Run \`codex login\`, then reopen Grok Bot.` |
| Codex refresh failed | `Codex login expired and could not be refreshed. Run \`codex login\` again.` |
| Claude Code query without a success result | `Claude Code ended without a result.` or joined `final.errors` |

Failed direct turns still persist an assistant `Router error: …` row so the shipped transcript can show the failure.

## Verification

Router identity and intercept wiring are asserted by:

- `tests/router-settings.test.mjs` — recovered overlay ids, default `cursor`, unknown preference falls back to `cursor`
- `tests/inference-router-transcript.test.mjs` — schemaVersion 2 `richText` round-trip; object `richText` dropped
- `tests/publication-packaging.test.mjs` — desktop RPC, `settings.json` sync, transcript merge methods, MCP bridge bind `127.0.0.1`, Codex Responses URL, OpenRouter base URL, 1200 ms composing delay

## Next

<CardGroup>
  <Card title="Choose an inference provider" href="/choose-inference-provider">
    Set Settings → Router, persist `inferenceProvider`, and satisfy Cursor, Claude Code, Codex, or OpenRouter credentials.
  </Card>
  <Card title="Provider clients" href="/provider-clients">
    Cursor default path, Claude Agent SDK query, Codex Responses, OpenRouter model `openai/gpt-5.2`.
  </Card>
  <Card title="Route Grok Bot plugin tools" href="/route-mcp-tools">
    `listRoutedMcpTools` and `executeRoutedMcpTool` across the Claude Code MCP bridge versus Codex/OpenRouter direct execution.
  </Card>
  <Card title="Settings schema" href="/settings-schema">
    `settings.json` version 1 fields including `inferenceProvider` and `inferenceRouterUsage`.
  </Card>
  <Card title="Desktop RPC" href="/desktop-rpc">
    `window.desktop.agent` `getInferenceRouter` / `setInferenceRouter` and secrets channels.
  </Card>
  <Card title="Router and sandbox failures" href="/router-failures">
    Unknown provider, private `auth.json`, missing Claude Code, missing `OPENROUTER_API_KEY`.
  </Card>
</CardGroup>