# Cabinet Daemon — WebSocket Bus, PTY Sessions & Job Scheduler

> The unified background server (server/cabinet-daemon.ts) that combines the WebSocket event bus, node-cron job scheduler, PTY/xterm terminal sessions with Claude lifecycle management, full-text search indexing, SQLite initialization, and file-watcher-driven real-time updates. Covers the daemon HTTP + WS server boot sequence, message protocol, and how each subsystem registers with the single server instance.

- Repository: hilash/cabinet
- GitHub: https://github.com/hilash/cabinet
- Human wiki: https://grok-wiki.com/public/wiki/hilash-cabinet-73c70f449a59
- Complete Markdown: https://grok-wiki.com/public/wiki/hilash-cabinet-73c70f449a59/llms-full.txt

## Source Files

- `server/cabinet-daemon.ts`
- `server/db.ts`
- `server/pty/manager.ts`
- `server/pty/claude-lifecycle.ts`
- `server/pty/types.ts`
- `server/search/index-builder.ts`
- `server/search/search-service.ts`

---

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [server/cabinet-daemon.ts](server/cabinet-daemon.ts)
- [server/db.ts](server/db.ts)
- [server/pty/manager.ts](server/pty/manager.ts)
- [server/pty/claude-lifecycle.ts](server/pty/claude-lifecycle.ts)
- [server/pty/types.ts](server/pty/types.ts)
- [server/search/index-builder.ts](server/search/index-builder.ts)
- [server/search/search-service.ts](server/search/search-service.ts)
- [server/search/watcher.ts](server/search/watcher.ts)
- [server/search/types.ts](server/search/types.ts)
</details>

# Cabinet Daemon — WebSocket Bus, PTY Sessions & Job Scheduler

The Cabinet Daemon (`server/cabinet-daemon.ts`) is the unified background server process for Cabinet. It combines every subsystem that must run continuously but outside the Next.js request cycle: a node-pty terminal multiplexer for AI agent sessions, a `node-cron` job scheduler for agent jobs and heartbeats, a WebSocket event bus for real-time frontend updates, an in-process FlexSearch full-text index for Markdown notes, and the SQLite database initialization layer. All subsystems share one `http.Server` instance and are wired together at startup.

Understanding the daemon is essential for tracing how a scheduled job fires, how a Claude terminal session is spawned and auto-completed, how the frontend receives live search-index updates, and how conversation state is finalized after a process exits.

---

## Daemon Boot Sequence

The daemon follows a strict sequential initialization before the HTTP server starts listening, and launches several async subsystems immediately after `server.listen` completes.

```text
1. ensureBetterSqlite3()        — native-binary preflight (rebuild if ABI mismatch)
2. loadCabinetEnv()             — merge .cabinet.env into process.env
3. getDb()                      — open SQLite, run file + SQL migrations
4. http.createServer(...)       — create HTTP server (handlers attached inline)
5. WebSocketServer (PTY)        — noServer, path: / or /api/daemon/pty
6. WebSocketServer (Events)     — noServer, path: /events or /api/daemon/events
7. server.listen(PORT)  ──────────────────────────────────────┐
   └→ reloadSchedules()         — parse .agents/*/persona.md + .jobs/*.yaml   │
   └→ cleanupStaleRunningConversations()                                        │
   └→ cleanupStaleStagingAttachments()                                          │
   └→ guardAgainstBigTree()                                                     │
       └→ bootstrapSearchIndex()                                                 │
           └→ startSearchWatcher()                                               │
   └→ loadExternalAdapters()                                                    │
   └→ telemetry: app.launched                                                   │
```

Sources: [server/cabinet-daemon.ts:1-25](), [server/cabinet-daemon.ts:1885-1940]()

---

## HTTP Server and API Endpoints

The daemon runs a plain `http.createServer` (no Express). Every route except `/health` requires a daemon bearer token validated by `isDaemonTokenValid`. CORS is applied per-request; only origins listed in `CABINET_APP_ORIGIN` or loopback addresses are allowed.

| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/health` | Status: session count, job count, subscriber count |
| `GET` | `/session/:id/output` | Poll output for active or completed session |
| `POST` | `/sessions` | Create a detached session without a WebSocket |
| `POST` | `/session/:id/input` | Write stdin to a live PTY session |
| `POST` | `/session/:id/close` | Graceful close: write `/exit`, 2s SIGTERM fallback |
| `POST` | `/session/:id/stop` | Immediate SIGTERM, SIGKILL fallback after 2 s |
| `GET` | `/sessions` | List all active sessions |
| `POST` | `/reload-schedules` | Force-reload all cron schedules |
| `POST` | `/trigger` | Manually spawn a one-off prompt session |
| `GET` | `/search` | Full-text search: pages, agents, tasks |

Sources: [server/cabinet-daemon.ts:1501-1790]()

### WebSocket Upgrade Routing

Two `WebSocketServer` instances share the same underlying `http.Server` via `noServer` mode. The `upgrade` event on the server dispatches by path:

- `/` or `/api/daemon/pty` → PTY terminal server (`wssPty`)
- `/events` or `/api/daemon/events` → event bus (`wssEvents`)

All upgrades are rejected with HTTP 401 if the token is invalid.

Sources: [server/cabinet-daemon.ts:1800-1840]()

---

## WebSocket Event Bus

The event bus (`/events`) provides a pub/sub broadcast channel for real-time updates from daemon subsystems to the frontend.

### Protocol

Clients connect and receive all channels by default (`channels = new Set(["*"])`). They can narrow subscriptions by sending JSON frames:

```json
// Subscribe to a specific channel
{ "subscribe": "search" }

// Unsubscribe
{ "unsubscribe": "search" }
```

The daemon calls `broadcast(channel, data)` from any subsystem. Each message is serialized as:
```json
{ "channel": "search", "type": "search:ready", "pages": 142, "tookMs": 87 }
```

### Event Channels

| Channel | Message types | Emitted by |
|---------|--------------|-----------|
| `search` | `search:indexing`, `search:ready`, `search:indexed`, `search:error` | `bootstrapSearchIndex`, `startSearchWatcher` |

Sources: [server/cabinet-daemon.ts:610-660](), [server/cabinet-daemon.ts:195-210]()

The `subscribers` array is managed directly:

```ts
// server/cabinet-daemon.ts
const subscribers: EventSubscriber[] = [];

function broadcast(channel: string, data: Record<string, unknown>): void {
  const message = JSON.stringify({ channel, ...data });
  for (const sub of subscribers) {
    if (sub.channels.has(channel) || sub.channels.has("*")) {
      if (sub.ws.readyState === WebSocket.OPEN) sub.ws.send(message);
    }
  }
}
```

---

## PTY Session Architecture

### Session Types

Every active terminal run lives in the `sessions: Map<string, ActiveSession>` map. Two kinds coexist:

```text
BaseSession (server/pty/types.ts)
├── PtySession  (kind: "pty")       — node-pty subprocess + xterm stream
└── StructuredSession (kind: "structured") — adapter.execute() async runner
```

The `PtySession` adds PTY-specific fields: the `node-pty` handle, timers for auto-exit, initial-prompt bookkeeping, `outputMode` (plain vs. `claude-stream-json`), and the `trigger` that controls whether auto-exit fires or the session stays alive awaiting user input.

Sources: [server/pty/types.ts:1-80]()

### Session Routing: `createSession`

The `createSession` function is the single entry point for spawning any session, regardless of whether it arrives from the WebSocket path or the HTTP `/sessions` endpoint:

```text
createSession(input)
    │
    ├── adapterType === "shell"       → ptyManager.spawn() (plain shell)
    ├── adapter.executionEngine !== "legacy_pty_cli"
    │       → createStructuredSession()  (native async adapter)
    └── legacy_pty_cli or no adapter → ptyManager.spawn() (PTY CLI)
```

Sources: [server/cabinet-daemon.ts:498-545]()

### PTY Manager (`server/pty/manager.ts`)

`createPtyManager(deps)` is a factory that receives shared infrastructure as `PtyManagerDeps` (the `sessions` map, `completedOutput` map, callbacks for finalization/output/cleanup) so it can operate without importing the full daemon module.

At spawn time, the manager:
1. Resolves the execution provider id and calls `getSessionLaunchSpec` or `getOneShotLaunchSpec` to obtain the `command`, `args`, and `readyStrategy`.
2. Merges `.cabinet.env` values at spawn time (mtime-cached, no daemon restart needed).
3. Calls `node-pty`'s `pty.spawn` with `xterm-256color`, 120×30 cols/rows, and an enriched `PATH` that includes `~/.local/bin`, `/opt/homebrew/bin`, and NVM node bins.
4. For one-shot Claude sessions, rewrites args to add `--output-format stream-json --include-partial-messages`.
5. For non-shell sessions, sets `ZDOTDIR=/dev/null` and clears `BASH_ENV` to suppress dotfile noise from sub-shells the CLI may spawn.

Sources: [server/pty/manager.ts:90-200]()

### Structured Sessions

When an adapter's `executionEngine` is not `"legacy_pty_cli"`, the daemon calls `adapter.execute()` directly — no PTY. The structured session:

- Captures `pid` + `processGroupId` from the `onSpawn` callback.
- Collects `stdout` via `onLog` into `session.output[]`; `stderr` is buffered separately for error classification and never shown to the user.
- On completion, extracts `adapterSessionId`, `adapterSessionParams`, `adapterUsage`, and runs `adapter.classifyError` to produce a typed `adapterErrorKind` (`cli_not_found`, `auth_expired`, `rate_limited`, etc.).
- Persists a resume handle via `writeSession` on successful exit so future turns can resume the same adapter session.

Sources: [server/cabinet-daemon.ts:300-490]()

### Reconnect and Replay

When a WebSocket client connects with `reconnect=1`, the daemon never spawns a new process. Instead it:
1. Looks up the in-memory `completedOutput` cache (recent runs).
2. Falls back to `readConversationTranscript` from disk for older sessions.
3. Emits a provenance banner (provider, adapter, timestamps) then the replay, then closes the socket.

Sources: [server/cabinet-daemon.ts:740-820]()

---

## Claude Lifecycle Management (`server/pty/claude-lifecycle.ts`)

Claude Code runs interactively as a persistent TUI. The lifecycle module manages the handshake between Cabinet and the Claude CLI for both one-shot (job/heartbeat) and manual sessions.

### Session State Machine

```mermaid
stateDiagram-v2
    [*] --> Spawned : ptyManager.spawn()
    Spawned --> WaitingForReady : await claudePromptReady()
    WaitingForReady --> PromptSubmitted : submitInitialPrompt()
    PromptSubmitted --> Running : output streaming
    Running --> AwaitingInput : trigger=manual AND\ntranscriptShowsCompletedRun\n[1.2s idle grace]
    Running --> Completing : trigger≠manual AND\ntranscriptShowsCompletedRun\n[1.2s grace timer]
    AwaitingInput --> Running : new output > 120 bytes\n[2s busy grace]
    Completing --> Exited : completeClaudeSession()\n→ write /exit → kill fallback
    Exited --> [*] : finalizeSessionConversation()
```

### Key Functions

| Function | Responsibility |
|----------|---------------|
| `claudePromptReady(output)` | Detects Claude's input cursor via `shift+tab to cycle` footer or bare `❯`/`>` glyph |
| `submitInitialPrompt(session)` | Writes prompt to PTY, then sends `\r` after 600ms to clear Claude's paste-grouping window |
| `consumeStructuredOutput(session, chunk)` | Passes chunks through `ClaudeStreamAccumulator` when `outputMode === "claude-stream-json"` |
| `maybeAutoExitClaudeSession(session, deps)` | After each chunk, checks `transcriptShowsCompletedRun`; schedules 1.2s completion timer or idle-flip depending on `trigger` |
| `completeClaudeSession(session, output, deps)` | Sets `resolvedStatus = "completed"`, calls `finalizeConversation`, writes `/exit\r`, sets 1.5s kill fallback |
| `scheduleStreamCabinetExtraction(session)` | Debounced (1s) extraction of fenced ` ```cabinet ` blocks for live `meta.summary` and `meta.artifactPaths` updates |
| `distillPtyOutput(plain, exitCode, providerId)` | Generates a synthetic one-liner summary from PTY output (avoids feeding TUI chrome to `parseCabinetBlock`) |

Sources: [server/pty/claude-lifecycle.ts:1-50](), [server/pty/claude-lifecycle.ts:90-190](), [server/pty/claude-lifecycle.ts:200-280]()

### Manual vs. Scheduled Trigger Behavior

The `trigger` field from `ConversationMeta` is threaded through to each `PtySession`. It changes the behavior at idle detection:

- **`trigger === "manual"`**: On idle, flip `meta.awaitingInput = true` and publish `task.updated`; keep the PTY alive so the user can continue typing in the xterm. When new output arrives (> 120 bytes over 2 s), flip back to "running".
- **Any other trigger** (`job`, `heartbeat`, `agent`): Start the 1.2 s grace timer and auto-exit. This prevents scheduled job runs from accumulating unclosed PTY processes.

Sources: [server/pty/claude-lifecycle.ts:240-310]()

---

## Job Scheduler

The scheduler uses `node-cron` to fire agent jobs and persona heartbeats at configured intervals. It discovers schedule configuration from the filesystem on boot and live-reloads on changes.

### Schedule Sources

| Config file | Parsed for |
|------------|-----------|
| `.agents/<slug>/persona.md` frontmatter | `heartbeat` (cron expr), `active`, `heartbeatEnabled` |
| `.jobs/<name>.yaml` | `id`, `schedule`, `enabled`, `ownerAgent`, `prompt`, `oneShot`, `timeout` |

Cabinet supports multiple "cabinet" data directories (rooms); `discoverAllCabinets()` enumerates all of them and each gets its own job/heartbeat namespace.

### Schedule Key Format

```
{cabinetPath}::job::{agentSlug}/{jobId}
{cabinetPath}::heartbeat::{agentSlug}
```

This ensures that jobs in different rooms never collide even if they share the same slug.

### Job Firing

When a cron task fires, `scheduleJob` calls the Next.js app API via `fetch`:

```ts
// server/cabinet-daemon.ts
void putJson(`${getAppOrigin()}/api/agents/${job.agentSlug}/jobs/${job.id}`, {
  action: "run",
  source: "scheduler",
  cabinetPath: job.cabinetPath,
  scheduledAt,
})
```

One-shot jobs are disabled via a follow-up `PUT` with `{ action: "update", enabled: false }` and removed from `scheduledJobs`.

Sources: [server/cabinet-daemon.ts:670-760]()

### Live Reload

A `chokidar` watcher monitors:
- `DATA_DIR/**/.agents/*/persona.md`
- `DATA_DIR/**/.jobs/*.yaml`
- `DATA_DIR/**/.agents/*/jobs/*.yaml`
- `DATA_DIR/**/.cabinet`

Any `add`, `change`, or `unlink` event queues a 200ms debounced `reloadSchedules()`. If the watcher itself fails with `EMFILE` or `ENOSPC`, it is closed gracefully and manual `POST /reload-schedules` remains available.

Sources: [server/cabinet-daemon.ts:1845-1880]()

---

## Full-Text Search Subsystem

### Index Architecture

The search index is built on [FlexSearch](https://github.com/nextapps-de/flexsearch) `Document` with `tokenize: "forward"` (prefix matching). Four fields are indexed per document:

| Field | Weight |
|-------|--------|
| `title` | 100 |
| `headings` | 50 |
| `tags` | 30 |
| `body` | 10 |

The `SearchIndex` class (`server/search/index-builder.ts`) wraps the FlexSearch document with a `Map<string, IndexedPageRecord>` side-store that preserves structured metadata (tagList, icon, modified, per-line content arrays) for snippet extraction.

Sources: [server/search/index-builder.ts:1-80]()

### Bootstrap Process

```ts
// server/cabinet-daemon.ts (startup sequence)
void guardAgainstBigTree().then(() =>
  bootstrapSearchIndex().then(() => startSearchWatcher())
);
```

`bootstrapSearchIndex` calls `walkDataDir()` to enumerate all `.md` files (following symlinked cabinet roots, skipping hidden directories and `CLAUDE.md`), then `buildPageRecord(fsPath, virtualPath)` for each file. Virtual paths strip the `DATA_DIR` prefix and `.md` extension; `index.md` files collapse to their parent directory path.

During indexing the daemon broadcasts `search:indexing` to all event-bus subscribers. On completion it broadcasts `search:ready` with `{ pages, tookMs }`.

Sources: [server/cabinet-daemon.ts:193-235](), [server/search/index-builder.ts:115-185]()

### Big-Tree Guard

Before bootstrapping the index, the daemon runs a preflight directory count. If the DATA_DIR contains more than 1,500 watchable directories **and** `CABINET_ALLOW_BIG_TREE` is not set, the daemon prints a diagnostic and calls `process.exit(1)`. This prevents exhausting OS file descriptor limits (`EMFILE`) that would crash the watcher silently.

Sources: [server/cabinet-daemon.ts:115-175]()

### Live Watcher

`startWatcher(searchIndex, opts)` (`server/search/watcher.ts`) opens a chokidar watcher on `DATA_DIR` for `*.md` file events. Each event is debounced by 150ms; then:

- `add` → `index.add(record)`
- `change` → `index.update(record)`
- `unlink` → `index.remove(virtualPath)`

Each successful update triggers the `onIndexed` callback, which the daemon uses to broadcast `search:indexed` on the event bus.

On `EMFILE`/`ENOSPC`, the watcher shuts itself down gracefully (logs once, no crash). Search queries still return results; live updates simply stop.

Sources: [server/search/watcher.ts:1-100]()

### Search Endpoint

`GET /search?q=...&scope=all|pages|agents|tasks&limit=50&cabinet=path` calls `runSearch(sources, query, scope, limit, cabinet)`. The response shape:

```ts
interface SearchResponse {
  query: string;
  scope: SearchScope;
  pages: PageHit[];    // up to 50 by default
  agents: AgentHit[];  // up to 20
  tasks: TaskHit[];    // up to 20
  tookMs: number;
  indexReady: boolean;
}
```

Agents and tasks are loaded on demand from filesystem/DB via `loadAgentDocs()` / `loadTaskDocs()`. Room scoping restricts results to paths within the requested `cabinet` prefix.

Sources: [server/cabinet-daemon.ts:1750-1795](), [server/search/search-service.ts:95-200]()

---

## SQLite Initialization (`server/db.ts`)

`getDb()` is a lazy singleton. On first call it:

1. Ensures `DATA_DIR` exists.
2. Runs `runFileMigrationsSync()` for filesystem-level migrations (renames, moves).
3. Opens `DATA_DIR/.cabinet.db` with `better-sqlite3`.
4. Sets `PRAGMA journal_mode = WAL` and `PRAGMA foreign_keys = ON`.
5. Runs numbered SQL migrations from `server/migrations/NNN_description.sql`.

`closeDb()` is called by the `shutdown` handler on SIGINT/SIGTERM.

Sources: [server/db.ts:1-60]()

---

## Graceful Shutdown

The `shutdown()` function handles both `SIGINT` and `SIGTERM`:

1. Emits `app.exited` telemetry.
2. Clears the telemetry session id.
3. Stops all `scheduledJobs` and `scheduledHeartbeats` cron tasks.
4. Sends `SIGTERM` to every active session (PTY and structured).
5. Closes the schedule file watcher.
6. Closes the SQLite connection.
7. Closes the HTTP server and calls `process.exit(0)`.

Sources: [server/cabinet-daemon.ts:1955-1985]()

---

## Session Memory Management

Two cleanup intervals run in the background:

| Interval | Purpose |
|----------|---------|
| Every 5 minutes | Evict `completedOutput` entries older than 30 minutes |
| Every 60 seconds | Move exited + detached sessions idle for > 10 minutes to `completedOutput`, then delete from `sessions` |

On daemon start, `cleanupStaleRunningConversations()` scans all cabinets for conversations still marked `"running"` in their `meta.json` (from a previous crashed daemon) and finalizes them as `"failed"`. This prevents permanently stuck spinners in the UI.

Sources: [server/cabinet-daemon.ts:405-430](), [server/cabinet-daemon.ts:855-870]()

---

## Summary

The Cabinet Daemon is a single Node.js process (`npx tsx server/cabinet-daemon.ts`) that owns the runtime surface for every background operation: it is the PTY multiplexer for Claude and other AI CLIs, the cron runner for agent jobs and persona heartbeats, the WebSocket broadcast bus for live frontend updates, the boot owner of the SQLite database, and the search indexer for the Markdown workspace. All subsystems share one `http.Server` and one `sessions` map, with WebSocket routing split between a PTY path and an event-bus path. The Claude lifecycle helpers in `server/pty/claude-lifecycle.ts` implement the nuanced handshake — prompt injection, stream parsing, cabinet-block extraction, and trigger-aware auto-exit — that distinguishes scheduled AI runs from interactive terminal sessions.