# Wire protocol v1

> HTTP-over-Tailscale POST `/sync`, AES-256-GCM seal, `SyncEnvelope` JSON fields, sink validation order, and the response semantics (401/400/409).

- Repository: mvanhorn/agentcookie
- GitHub: https://github.com/mvanhorn/agentcookie
- Human docs: https://grok-wiki.com/public/docs/mvanhorn-agentcookie-137da38edfae
- Complete Markdown: https://grok-wiki.com/public/docs/mvanhorn-agentcookie-137da38edfae/llms-full.txt

## Source Files

- `docs/protocol.md`
- `internal/protocol/envelope.go`
- `internal/protocol/sequence.go`
- `internal/protocol/allowlist.go`
- `internal/transport/crypto.go`

---

---
title: "Wire protocol v1"
description: "HTTP-over-Tailscale POST `/sync`, AES-256-GCM seal, `SyncEnvelope` JSON fields, sink validation order, and the response semantics (401/400/409)."
---

The agentcookie wire protocol is a single HTTP request: the source POSTs an AES-256-GCM-sealed `SyncEnvelope` to `http://<sink-tailnet-ip>:<port>/sync` over the Tailscale tailnet, with `Content-Type: application/octet-stream`. The sink decrypts, validates a strict sequence (`SyncEnvelope.protocol_version`, monotonic `sequence`, blocklist filter), applies the payload to Chrome state + sidecar + adapters + the secrets bus, and responds with a plain-text body summarizing what landed. Versioning is in-band: sources always emit the highest version they speak (`protocol.Version = 2`), sinks accept any version in `[protocol.MinVersion, protocol.Version]` (currently `1..2`); v1 envelopes simply leave the v2-introduced tarball and secrets fields nil.

## Layers, outside to inside

```text
+----------------------------------------------------------+
| Transport: HTTP/1.1 over Tailscale tailnet (100.x:port) |
|   POST /sync   Content-Type: application/octet-stream    |
+----------------------------------------------------------+
| Seal:       AES-256-GCM (12-byte nonce || ciphertext || tag) |
|   key = SHA-256(secret),  secret = pairing-derived 32 B  |
|                          OR legacy security.shared_secret |
+----------------------------------------------------------+
| Envelope:   JSON SyncEnvelope                            |
|   protocol_version, source_hostname, sequence,           |
|   cookies[], local_storage_tarball?, indexed_db_tarball?,|
|   indexed_db_skipped?, secrets?                          |
+----------------------------------------------------------+
```

The transport address is enforced to be a Tailscale CGNAT IP (`100.64.0.0/10`) at install/config time; `sink.yaml`'s `listen.addr` may not be empty and the wizard writes the host's detected 100.x address. The sink also exposes `GET /healthz` returning `ok`.

## Seal: AES-256-GCM with SHA-256-derived key

The transport seal is `internal/transport.SealWithSecret` / `OpenWithSecret`:

- Key derivation: `key = sha256.Sum256([]byte(secret))`, producing a 32-byte AES-256 key. SHA-256 is applied unconditionally, even when the input is already a uniformly random 32-byte pairing-derived key, so v0.11 sinks and v0.12+ sources interoperate on a single derivation path.
- Cipher: `cipher.NewGCM(aes.NewCipher(key))`, 12-byte nonce, 16-byte GCM tag, no additional data.
- Wire encoding: a fresh nonce is generated per envelope and prepended to the ciphertext: `nonce(12) || ciphertext || tag(16)`. The whole blob is the HTTP request body.
- Failure mode: any tampering or wrong-secret payload fails the AEAD tag and `OpenWithSecret` returns `"decrypt (wrong secret or tampered payload): ..."`, which the sink converts to `401 Unauthorized`.

Two key sources are supported, both feeding the same `secret` parameter:

| Source | Where | Notes |
| --- | --- | --- |
| Pairing-derived | `~/.config/agentcookie/keys/<peer>.json` | X25519 + HKDF-SHA256 handshake, 32 raw bytes. Canonical. |
| Legacy shared secret | `security.shared_secret` in `source.yaml` / `sink.yaml` | Must be ≥ 32 bytes; rejected at config load otherwise. |

## SyncEnvelope JSON

`internal/protocol.SyncEnvelope` is the JSON shape inside the seal:

<ResponseField name="protocol_version" type="int" required>
  The wire version. Source emits `protocol.Version` (currently `2`). Sink accepts any version in `[MinVersion, Version]` (currently `1..2`); anything outside is rejected `400`. v1 envelopes are valid: the v2-only fields decode as nil/empty.
</ResponseField>

<ResponseField name="source_hostname" type="string" required>
  The source's announced hostname (`pairing.LocalHostname()`). The sink uses this as the key in its replay-defense tracker and matches it to the paired key file under `~/.config/agentcookie/keys/`.
</ResponseField>

<ResponseField name="sequence" type="int64" required>
  Monotonically increasing per source. The source emits `time.Now().UnixNano()` so rapid syncs do not collide. The sink rejects any value not strictly greater than the highest accepted value for `source_hostname`.
</ResponseField>

<ResponseField name="cookies" type="array<chrome.Cookie>" required>
  One element per cookie. Each cookie carries `host_key`, `name`, `value` (plaintext after the source's Safe Storage decrypt), `path`, `expires_utc`, `is_secure`, `is_httponly`, `last_access_utc`, `has_expires`, `is_persistent`, `priority`, `samesite`, `source_scheme`, `source_port`. All fields are strings or integers; values are never raw bytes.
</ResponseField>

<ResponseField name="local_storage_tarball" type="bytes (base64 in JSON)">
  v2-only. Packed LevelDB tarball of Chrome's `Local Storage/leveldb` (`internal/chromedirsync.Pack`). Omitted on v1 envelopes; sink unpacks via `chromedirsync.Unpack` + `AtomicReplaceDir`.
</ResponseField>

<ResponseField name="indexed_db_tarball" type="bytes (base64 in JSON)">
  v2-only, opt-in via `AGENTCOOKIE_SYNC_INDEXEDDB=1`. Per-origin LevelDB directories larger than 5 MB are skipped at pack time.
</ResponseField>

<ResponseField name="indexed_db_skipped" type="string[]">
  v2-only. The origins skipped by the IndexedDB packer's size cap, surfaced for visibility.
</ResponseField>

<ResponseField name="secrets" type="map<cli, map<key, value>>">
  v2-only (`v0.13`). The secrets-bus payload, one map per registered CLI under `~/.agentcookie/secrets/`, post-filter against the manifest's sync policy. v0.12 sinks deserialize unchanged because the field is `omitempty`.
</ResponseField>

The Go declaration in `internal/protocol/envelope.go`:

```go
type SyncEnvelope struct {
    ProtocolVersion     int             `json:"protocol_version"`
    SourceHostname      string          `json:"source_hostname"`
    Sequence            int64           `json:"sequence"`
    Cookies             []chrome.Cookie `json:"cookies"`
    LocalStorageTarball []byte          `json:"local_storage_tarball,omitempty"`
    IndexedDBTarball    []byte          `json:"indexed_db_tarball,omitempty"`
    IndexedDBSkipped    []string        `json:"indexed_db_skipped,omitempty"`
    Secrets             map[string]map[string]string `json:"secrets,omitempty"`
}
```

A v1 envelope is literally the same struct with only the first four fields populated.

## Sink validation order

The `/sync` handler walks the envelope through a fixed sequence; each step rejects on failure with a status code that uniquely identifies the layer that failed.

```mermaid
sequenceDiagram
    participant Src as Source
    participant Net as Tailscale tailnet
    participant H as Sink /sync handler
    participant Seq as SequenceTracker (sequence.json)
    participant BL as BlocklistMatcher
    participant Sk as Sink writers

    Src->>Net: POST /sync (sealed body, octet-stream)
    Net->>H: HTTP request
    H->>H: Method check (POST only -> 405 otherwise)
    H->>H: LimitedReader cap (default 256 MiB)
    H->>H: io.ReadAll(body) -> 400 on read error
    H->>H: transport.OpenWithSecret -> 401 on AEAD fail
    H->>H: json.Unmarshal(SyncEnvelope) -> 400 on parse fail
    H->>H: ProtocolVersion in [MinVersion, Version] -> 400 otherwise
    H->>Seq: Accept(source_hostname, sequence)
    Seq-->>H: false (<=last seen) -> 409
    Seq-->>H: true (persist write-through)
    H->>BL: Filter(cookies)
    BL-->>H: passed[], droppedHosts{}
    H->>Sk: writers (sqlite/leveldb, sidecar, CDP, adapters, secrets)
    Sk-->>H: writeResult or err -> 500 on write fail
    H-->>Src: 200 OK + summary line
```

Step by step, in handler order (`internal/cli/sink.go`):

<Steps>
  <Step title="Method gate">
    Anything other than `POST` returns `405 Method Not Allowed` with body `POST only`.
  </Step>
  <Step title="Body cap">
    `httpserver.LimitedReader` wraps `r.Body` with `http.MaxBytesReader(..., MaxBodyBytes)`. Default cap for the `SinkSync` profile is `256 * 1024 * 1024` bytes (256 MiB), configurable via `sink.yaml`. Reads beyond the cap return `400 Bad Request` with `read body: http: request body too large`.
  </Step>
  <Step title="AEAD open">
    `transport.OpenWithSecret(sealed, transportSecret)`. Any failure (truncated body, wrong nonce length, wrong key, tamper) returns `401 Unauthorized` with `open payload: ...`.
  </Step>
  <Step title="JSON unmarshal">
    Parse failure returns `400 Bad Request` with `unmarshal envelope: ...`.
  </Step>
  <Step title="Protocol version range">
    `envelope.ProtocolVersion` must satisfy `MinVersion <= v <= Version`. Out-of-range returns `400 Bad Request` with `protocol version mismatch: got N, sink speaks M-N`.
  </Step>
  <Step title="Replay defense">
    `seqTracker.Accept(envelope.SourceHostname, envelope.Sequence)` requires the value be strictly greater than the highest accepted for that source. Rejection returns `409 Conflict` with `sequence N not greater than last seen for "<host>" (replay defense)`. State is persisted to `~/.agentcookie/sequence.json` (mode 0600, atomic rename) on every accept; a failed persist rolls back the in-memory update and rejects the request.
  </Step>
  <Step title="Sink-side blocklist filter">
    `BlocklistMatcher.Filter(cookies)` returns the passed cookies and a per-host drop count. An empty blocklist (sync-all default) drops nothing. Patterns use SQLite LIKE semantics (`%` wildcard), case-insensitive, against `host_key` (which carries a leading dot for subdomain cookies). Drops are logged on stderr; the count is included in the response body.
  </Step>
  <Step title="Apply payload">
    Cookies write via the SQLite/LevelDB path or, when `skip_chrome_sqlite: true`, the sidecar-only path (`~/.agentcookie/cookies-plain.db`). When `cdp.enabled` is set the sink also spawns headless Chrome and pushes via `Storage.setCookies`; CDP failures are logged but do not fail the sync because the sidecar write already succeeded. PP-CLI adapters run after the cookie write. If `envelope.Secrets` is non-empty the secrets-bus writer persists per-CLI `secrets.env` files. Any unrecoverable write returns `500 Internal Server Error` with `apply envelope: ...`.
  </Step>
</Steps>

<Note>
  The validation order is load-bearing for the response code contract: a 401 means the seal failed, a 400 means a structurally invalid envelope (unparsable, wrong version), and a 409 means a sequence that would re-open the replay window. Callers that retry should retry only on 5xx and transient transport errors — 4xx codes mean a coordinated source/sink fix is required.
</Note>

## Response semantics

The sink emits plain-text responses (`http.Error` for failures, `fmt.Fprintf(w, ...)` for success). The status code is the wire contract; the body is human-readable diagnostics.

| Status | When | Body shape |
| --- | --- | --- |
| `200 OK` | Envelope accepted and applied | `ok: wrote N cookies (M sidecar), L localStorage origins, I indexedDB origins; dropped D non-allowlisted cookies\n` |
| `200 OK` (dry-run) | `--dry-run` sink, envelope accepted | `dry-run ok: accepted N cookies; dropped D non-allowlisted\n` |
| `400 Bad Request` | Body read error, JSON parse error, or `protocol_version` out of `[MinVersion, Version]` | `read body: ...`, `unmarshal envelope: ...`, or `protocol version mismatch: got N, sink speaks M-N\n` |
| `401 Unauthorized` | AEAD open failed (wrong key, tampered payload, short body) | `open payload: decrypt (wrong secret or tampered payload): ...\n` |
| `405 Method Not Allowed` | Anything other than POST | `POST only\n` |
| `409 Conflict` | `sequence` not strictly greater than last seen for `source_hostname` | `sequence N not greater than last seen for "<host>" (replay defense)\n` |
| `500 Internal Server Error` | Apply step failed (SQLite/LevelDB write, sidecar write, etc.) | `apply envelope: ...\n` |

The source treats any non-200 status as a hard error and returns it from the per-tick push (`sink returned <status>: <body>`), surfacing it to `agentcookie status` and the source log. The source does not retry inside the push; the next watcher tick is the natural retry surface.

## Replay defense persistence

`SequenceTracker` is process-wide and write-through. The on-disk store is `~/.agentcookie/sequence.json`, mode `0600`, written atomically (`os.CreateTemp` + `os.Rename`). The JSON is a flat `{ "source_hostname": last_seq, ... }` map.

| Behavior | Outcome |
| --- | --- |
| File absent on boot | Empty map; first envelope of any sequence wins. |
| File corrupt on boot | Sink refuses to start with `parse sequence state ...: ... (delete this file to reset replay-defense state)`. |
| Save fails during `Accept` | In-memory update rolled back; `Accept` returns false; handler returns `409`. |
| Operator reset | Delete `~/.agentcookie/sequence.json` (only do this if you accept the brief replay window). |

Because `Sequence` is sourced at nanosecond granularity (`time.Now().UnixNano()`), rapid back-to-back syncs do not collide. Legacy 1-second-granularity sources still interoperate because the tracker requires only strict monotonic increase.

## Source side: how an envelope is constructed

The source push path (`internal/cli/source.go`) assembles, seals, and POSTs in this exact order:

```go
envelope := protocol.SyncEnvelope{
    ProtocolVersion:     protocol.Version,
    SourceHostname:      pairing.LocalHostname(),
    Sequence:            time.Now().UnixNano(),
    Cookies:             all,
    LocalStorageTarball: lsTarball,    // v2; nil for v1-shaped envelopes
    IndexedDBTarball:    idbTarball,   // v2; opt-in
    IndexedDBSkipped:    idbSkipped,   // v2
}
if secretsPayload != nil && len(secretsPayload.CLIs) > 0 {
    envelope.Secrets = secretsPayload.CLIs
}
payload, _ := json.Marshal(envelope)
sealed, _  := transport.SealWithSecret(payload, secret)

req, _ := http.NewRequestWithContext(postCtx, "POST", cfg.Sink.URL, bytes.NewReader(sealed))
req.Header.Set("Content-Type", "application/octet-stream")
resp, _ := httpserver.Client(httpserver.SyncClient).Do(req)
```

The `SyncClient` profile bounds the POST at a 5-minute client timeout to accommodate large LevelDB tarballs over a slow tailnet link.

## Versioning posture

- v1 is the original envelope: `protocol_version`, `source_hostname`, `sequence`, `cookies`. It remains wire-compatible. A v1-only client may POST envelopes with the other fields omitted.
- v2 adds `local_storage_tarball`, `indexed_db_tarball`, `indexed_db_skipped` (v0.7), and `secrets` (v0.13), all as `omitempty` fields.
- A breaking change bumps both `MinVersion` and `Version`, drops the prior version from the accepted range, and requires a coordinated source/sink upgrade. Out-of-band fields land first as additional optional v1/v2 envelope fields and may later graduate to required in a future v3.
- Listeners are tailnet-only: `sink.yaml`'s `listen.addr` must be set explicitly and the wizard derives it from `tsclient.RequireTailnetIP` (a 100.64.0.0/10 address). Pre-v0.12 silent fall-through to `127.0.0.1:9999` is intentionally removed.

## Replay window edge cases

<AccordionGroup>
  <Accordion title="Sink restart">
    Sequence state is persisted, so a captured payload cannot be replayed across a sink restart — the high-water marks for every `source_hostname` survive in `sequence.json`.
  </Accordion>
  <Accordion title="Source restart">
    The source sets `Sequence = time.Now().UnixNano()` on every push; restarts keep the value monotonically increasing because wall-clock nanoseconds keep advancing. No source-side state is required.
  </Accordion>
  <Accordion title="Clock skew">
    The tracker requires strict monotonic increase per source, not wall-clock truth. Any source that hands out monotonically increasing int64 values is acceptable; the sink does not interpret `Sequence` as time.
  </Accordion>
  <Accordion title="Multiple sources">
    Each `source_hostname` has its own high-water mark in the tracker, so two paired laptops with overlapping sequence ranges are tracked independently.
  </Accordion>
</AccordionGroup>

## Related pages

<CardGroup>
  <Card title="Pairing and per-peer keys" href="/pairing-and-keys">
    Where the AES-GCM `secret` comes from: X25519 + HKDF-SHA256 handshake and `~/.config/agentcookie/keys/<peer>.json`.
  </Card>
  <Card title="Configure source and sink" href="/configure-source-sink">
    `sink.yaml` `listen.addr` validation, `peer.hostname`, `skip_chrome_sqlite`, and the CDP toggle.
  </Card>
  <Card title="Source and sink topology" href="/source-sink-topology">
    How the source watcher and the sink listener fit on either side of `/sync`.
  </Card>
  <Card title="Secrets bus" href="/secrets-bus">
    The `secrets` envelope field, the per-CLI `secrets.env` layout, and the v2 adoption tiers.
  </Card>
  <Card title="Troubleshooting" href="/troubleshooting">
    Pairing `connection refused`, 401/400/409 walkthroughs, and the sink Keychain prompt FAQ.
  </Card>
</CardGroup>