# Collaboration server

> Hocuspocus/Yjs CRDT server lifecycle, per-project `ok start` locks, server-free MCP reads (`exec`, `preview_url`) versus server-routed writes, and protocol version boundaries.

- Repository: sashimikun/open-knowledge
- GitHub: https://github.com/sashimikun/open-knowledge
- Human docs: https://grok-wiki.com/public/docs/sashimikun-open-knowledge-5c45105c876e
- Complete Markdown: https://grok-wiki.com/public/docs/sashimikun-open-knowledge-5c45105c876e/llms-full.txt

## Source Files

- `packages/server/src/index.ts`
- `packages/cli/src/commands/start.ts`
- `packages/cli/src/commands/mcp.ts`
- `packages/cli/src/commands/lock-state.ts`
- `packages/server/src/mcp/tools/exec.ts`
- `packages/server/src/mcp/tools/shared.ts`
- `packages/core/src/protocol-version.ts`

---

---
title: "Collaboration server"
description: "Hocuspocus/Yjs CRDT server lifecycle, per-project `ok start` locks, server-free MCP reads (`exec`, `preview_url`) versus server-routed writes, and protocol version boundaries."
---

Open Knowledge runs one **collaboration server per project**. That process hosts a Hocuspocus/Yjs CRDT layer for real-time editing, an HTTP API for agent writes and search, and (when not in ephemeral mode) an HTTP MCP endpoint at `/mcp`. The editor UI is a separate sibling process (`ok ui`) that connects over WebSocket; agents typically reach the server through MCP, which may auto-start it on demand.

## Architecture

The server is built on **Hocuspocus** with **Yjs** documents. Each markdown file maps to a Y.Doc; a persistence extension debounces writes back to disk and coordinates with the shadow-repo timeline. Extensions also maintain live derived indexes (backlinks, tags), agent sessions, and CC1 config broadcast.

```mermaid
flowchart TB
  subgraph clients [Clients]
    Editor["Editor UI (`ok ui`)"]
    Agent["Agent MCP client"]
    Desktop["OK Desktop"]
  end

  subgraph server [Collaboration server (`ok start`)]
    HTTP["HTTP server"]
    WS["WebSocket `/collab`"]
    Hocus["Hocuspocus + Yjs"]
    Persist["Persistence extension"]
    MCP["HTTP MCP `/mcp`"]
    API["HTTP API `/api/*`"]
  end

  subgraph disk [Filesystem]
    Content["Content dir (`content.dir`)"]
    Local["`.ok/local/` locks + state"]
  end

  Editor --> WS
  Agent --> MCP
  Desktop --> HTTP
  WS --> Hocus
  MCP --> API
  API --> Hocus
  Hocus --> Persist
  Persist --> Content
  server --> Local
```

| Surface | Path / entry | Purpose |
| --- | --- | --- |
| CRDT sync | WebSocket `/collab` | Editor and in-process agent sessions |
| Keepalive | WebSocket `/collab/keepalive` | MCP session presence (loopback only) |
| Agent HTTP API | `/api/*` | Writes, search, documents, share, local-ops |
| MCP over HTTP | `/mcp` | Streamable HTTP MCP when server is running |
| Global MCP stdio | `ok mcp` | Per-call project routing; proxies to `/mcp` or auto-starts server |

Loopback and host-header checks apply to `/mcp` and collaboration sockets. The React editor shell is **not** served by the collab server unless you pass `--react-shell-dist-dir`; normally `ok start` auto-spawns `ok ui` as a sibling.

## Server lifecycle

### Start

<Steps>
<Step title="Prerequisites">

Run `ok init` so the project has `.ok/config.yml`. From the project root:

```bash
ok start
```

Use `ok start --open` to open the editor URL after the server is ready.

</Step>

<Step title="Boot sequence">

`ok start` calls `bootStartServer`, which:

1. Verifies the project scaffold (unless `--single-file` ephemeral mode).
2. Runs reclaim sweeps for MCP configs, launch.json, and skills.
3. Boots the collab server via `bootServer` → `createServer` (acquires `server.lock`, asserts state manifest compatibility, constructs Hocuspocus).
4. Mounts HTTP handlers (`/mcp`, `/api/*`, optional content assets).
5. Listens on the chosen host/port and writes the bound port into `server.lock`.
6. Optionally auto-spawns `ok ui` when no live `ui.lock` exists.

Default idle shutdown threshold is **30 minutes** with no WebSocket clients on `/collab`.

</Step>

<Step title="Verify">

```bash
ok status
```

Expect `server` and `ui` entries showing `alive` with pid and port. The start banner prints the editor URL and API URL.

</Step>
</Steps>

<ParamField body="--port" type="number">
Collab server listen port. Also respected via `PORT` env unless `--ui-port` is set (worktree-preview connect path).
</ParamField>

<ParamField body="--ui-port" type="number">
Pin the UI sibling to a port. If a server already holds the project lock, `ok start --ui-port <n>` connects a UI sibling instead of starting a second server.
</ParamField>

<ParamField body="--host" type="string">
Bind address. Defaults to `localhost`; overridable via `HOST` env.
</ParamField>

<ParamField body="--single-file" type="string">
Ephemeral mode: scope the server to one markdown file. Git and MCP HTTP are off; useful for quick notes without a full project.
</ParamField>

### Lock kinds

When the server acquires `server.lock`, it records a **kind**:

| Kind | Set by | Typical behavior |
| --- | --- | --- |
| `interactive` | `ok start`, desktop | Stays up until you quit or `ok stop` |
| `mcp-spawned` | MCP auto-start (`OK_LOCK_KIND=mcp-spawned`) | Subject to idle shutdown (~30 min with no `/collab` clients) |

Collision errors are tailored: an `interactive` lock means the desktop or CLI server is already running; an `mcp-spawned` lock suggests waiting for idle shutdown or running `ok stop`.

### Idle shutdown

`attachIdleShutdown` watches WebSocket upgrades on `/collab`. When the client count hits zero, a timer starts (default 30 min, warn 5 min before). On fire, the handler SIGTERM/SIGKILLs the UI sibling if needed, then runs `destroy()`.

MCP-spawned servers rely on this path to release locks without manual `ok stop`.

### Graceful shutdown

On SIGINT/SIGTERM, `ok start` prints a shutdown notice and calls `destroy()`, which sequentially:

1. Detaches idle-shutdown watcher
2. Shuts down mount layer (WebSocket server, keepalive timers)
3. Closes MCP HTTP handler
4. Closes HTTP server
5. Destroys Hocuspocus (flushes Y.Doc persistence)
6. Releases `ui.lock` if owned
7. Flushes telemetry and log sinks

Each step has a 5 s timeout; partial failures are aggregated but the lock is still released when the owning process exits cleanly.

## Per-project locks

Runtime state lives under **`.ok/local/`** (the lock directory). Two JSON lock files coordinate processes:

:::files
.ok/local/
├── server.lock    # Collaboration server (pid, port, kind, versions)
├── ui.lock        # Editor UI sibling (pid, port)
├── state.json     # State manifest (schema + version metadata)
└── last-spawn-error.log
:::

### Lock metadata

Each lock file stores:

| Field | Meaning |
| --- | --- |
| `pid` | Owning process |
| `hostname` | Machine that created the lock |
| `port` | Bound HTTP port (0 until listen completes) |
| `startedAt` | ISO timestamp |
| `worktreeRoot` | Project directory the lock covers |
| `kind` | `interactive` or `mcp-spawned` (server only) |
| `protocolVersion` | Integer collab protocol version (currently `1`) |
| `runtimeVersion` | Package semver of the owning binary |

### Lock inspection states

`ok status` and `ok stop` classify locks through `inspectLock`:

| Status | Meaning | Action |
| --- | --- | --- |
| `missing` | No lock file | Safe to start |
| `alive` | Local host, live pid | Server/UI running |
| `dead-pid` | Stale lock | Run `ok clean` |
| `corrupt` | Unparseable JSON | Run `ok clean` |
| `foreign-host` | Different hostname | Cannot verify locally; `ok stop` may still signal remote pid |

`ok stop` sends SIGTERM to alive `server` and `ui` targets. `ok clean` removes stale locks. `ok ps` lists locks across discovered projects.

## MCP routing: server-free reads vs server-routed writes

MCP tools split into two classes based on whether they need a live Hocuspocus instance.

### Server-free (no collab server required)

These tools work against the filesystem and project config. They do **not** call `/api/*` and do not mutate Yjs state.

| Tool | Behavior without server |
| --- | --- |
| `exec` | Read-only bash allowlist (`cat`, `grep`, `ls`, …) over content; enriches paths with frontmatter, backlinks (batch fetch is optional when server URL resolves), shadow-repo activity |
| `workflow` | Returns role-specific prose frames; `previewUrl: null` |
| `config` | Reads merged YAML config scopes |
| `palette` | Lists editor command palette entries from config |

`exec` enforces a security invariant: if content mtimes change during a read-only call, the tool fails with `security_invariant_violation`.

For literal-string search without embeddings, prefer `exec({ command: "grep -rn …" })` over `search` when the server is down.

### Server-routed (Hocuspocus required)

These tools resolve a server URL and POST/PUT/DELETE to HTTP API routes. Without a running server they return:

```
Error: Hocuspocus server is not running. Start it with `ok start`, then retry.
For disk-only writes without real-time sync, use your native Edit tool directly.
```

| Tool | Needs server for |
| --- | --- |
| `write`, `edit`, `delete`, `move` | Yjs-backed document mutations + disk persistence |
| `search` | Semantic / indexed search via `/api/search` |
| `links` | Link graph queries |
| `history`, `checkpoint`, `restore_version` | Shadow-repo timeline |
| `conflicts`, `resolve_conflict` | Merge conflict state |
| `share_link` | Share URL generation |

### `preview_url` — special case

`preview_url` is not a CRDT write, but it **can auto-start** the collab server (same `OK_MCP_AUTOSTART` gate as write tools) to bring up the UI and return a browser-reachable URL. Per-response `previewUrl` fields on `exec`/`write`/`edit` are **route-only** (`/#/<doc>`); call `preview_url` for a full openable URL.

When the server runs but no UI has bound, the tool returns a hint to retry or run `ok ui`.

### Global MCP (`ok mcp`) routing

The global stdio MCP server registers all tools with a **per-call** `serverUrl` resolver:

1. Resolve project root from explicit `cwd`, sticky session anchor, or the client's single MCP `roots` entry.
2. Read `server.lock`; if alive, return `http://localhost:<port>`.
3. If no server and `OK_MCP_AUTOSTART` is not `0`, spawn detached `ok start` with `OK_LOCK_KIND=mcp-spawned`, poll until port appears (default 5 s timeout, overridable via `OK_MCP_SPAWN_TIMEOUT_MS`).
4. Start a keepalive WebSocket per project so agent presence is visible in the editor.

<ParamField body="cwd" type="string" required={false}>
Absolute path inside an Open Knowledge project. Required for global MCP when the client advertises zero or multiple roots; optional when anchored to one project or exactly one root is advertised.
</ParamField>

<ParamField body="OK_MCP_AUTOSTART" type="env">
Set to `0` to disable auto-start. Write tools and `preview_url` then fail until you run `ok start` manually.
</ParamField>

<ParamField body="OK_MCP_SPAWN_TIMEOUT_MS" type="env">
Milliseconds to wait for auto-spawned server to bind (default `5000`).
</ParamField>

On macOS, `ok mcp` may proxy to the Desktop bundle's `ok` binary when installed; use `--no-bundle-proxy` or `OK_BUNDLE_PROXY=0` to run the npm-fetched server in-process.

### Per-project HTTP MCP shim

When the collab server is already running, editors can register MCP against `http://localhost:<port>/mcp` directly:

```bash
ok mcp --port 4321
```

This bridges stdio MCP to the HTTP endpoint without auto-start logic.

## Protocol version boundaries

Open Knowledge tracks three independent version numbers:

| Constant | Value | Role |
| --- | --- | --- |
| `PROTOCOL_VERSION` | `1` | Collab wire protocol; must match across clients and lock metadata |
| `STATE_SCHEMA_VERSION` | `1` | `.ok/local/state.json` manifest schema |
| `RUNTIME_VERSION` | package semver | npm/DMG release identity |

### State manifest (`.ok/local/state.json`)

On boot, `assertCompatibleStateManifest` either:

- **Refuses boot** if `stateSchemaVersion` is incompatible (no on-the-fly migration).
- **Writes a fresh manifest** for new projects.
- **Adopts** pre-versioned projects at schema `0` when a shadow repo already exists.

The manifest records `createdBy` and `lastWriteBy` with `runtimeVersion` and `protocolVersion`.

### Lock version gates

New locks auto-populate `protocolVersion` and `runtimeVersion`. A live lock missing either field is treated as **incompatible** (`missing-fields`) and will not be reused for routing decisions that require version metadata.

### Client ↔ server drift

HTTP and WebSocket clients send version headers (`x-ok-client-protocol`, `x-ok-client-runtime`, `x-ok-client-kind`). The desktop shell classifies attached servers:

| Drift | Result |
| --- | --- |
| `protocolVersion` differs | `older` or `newer` on **protocol** dimension — hard boundary |
| Same protocol, different runtime semver | `older` / `newer` on **runtime** dimension |
| Missing version fields in lock | `indeterminate` |

Protocol mismatches are the hard line: they indicate incompatible CRDT or API contracts. Runtime drift is informational (semver compare) unless versions are unresolvable.

## Operational commands

| Command | Purpose |
| --- | --- |
| `ok start` | Boot collab server (+ auto UI sibling) |
| `ok stop` | SIGTERM server and UI from lock pids |
| `ok status` | Human-readable lock report |
| `ok ps` | List locks across projects |
| `ok clean` | Remove stale/corrupt locks |
| `ok diagnose health` | Include server-lock health check |

<RequestExample>

```bash
ok start --open
```

</RequestExample>

<ResponseExample>

```
open-knowledge 0.x.x

  Editor   http://localhost:5173
  API      http://localhost:4321

  Open the Editor URL in your browser to start editing.
```

</ResponseExample>

## Failure modes

<AccordionGroup>
<Accordion title="Server lock collision">

Another process holds `server.lock` with a live pid. Run `ok status` to see pid/port/kind. Use `ok stop`, quit the desktop app, or `--cwd` to point at a different project. For `mcp-spawned` locks, wait for idle shutdown or stop explicitly.

</Accordion>

<Accordion title="MCP write fails with Hocuspocus not running">

The global MCP resolver could not find or spawn a server. Start manually with `ok start`, or ensure `OK_MCP_AUTOSTART` is not `0`. Check `.ok/local/last-spawn-error.log` after failed auto-start.

</Accordion>

<Accordion title="Stale or corrupt locks">

`ok status` reports `dead-pid` or `corrupt`. Run `ok clean` before restarting.

</Accordion>

<Accordion title="State manifest incompatible">

Boot aborts with `StateManifestError` when `stateSchemaVersion` does not match the binary. Upgrade/downgrade the Open Knowledge package to a compatible release; on-the-fly migration is intentionally out of scope.

</Accordion>

<Accordion title="Degraded subsystems at start">

After `ready`, `ok start` may warn about degraded components (`shadow-repo`, `file-watcher`, `head-watcher`). The server stays up but version history, external file sync, or branch-switch safety may be limited — check server logs.

</Accordion>

<Accordion title="exec security invariant violation">

A read-only `exec` call detected content file mutations. This indicates a parser bug; do not trust the output. Report via `ok diagnose bundle`.

</Accordion>
</AccordionGroup>

## Related pages

<Card href="/quickstart" title="Quickstart" icon="rocket">
Run `ok init`, `ok start --open`, and verify MCP tools respond.
</Card>

<Card href="/mcp-tools-reference" title="MCP tools reference" icon="wrench">
Full tool catalog, input nesting, and preview envelope semantics.
</Card>

<Card href="/http-api-reference" title="HTTP API reference" icon="globe">
Agent and editor routes the server-routed MCP tools call.
</Card>

<Card href="/cli-reference" title="CLI reference" icon="terminal">
`start`, `stop`, `status`, `ps`, and `clean` flags and behavior.
</Card>

<Card href="/troubleshooting" title="Troubleshooting" icon="life-buoy">
Lock diagnosis, `ok diagnose health`, and crash recovery with `ok clean`.
</Card>

<Card href="/wire-agent-editors" title="Wire agent editors" icon="plug">
Register MCP, verify with `exec`, and configure auto-start behavior.
</Card>