# Context compaction

> Auto and manual compaction triggers, reasons, retries on stream drop, and interactions with in-flight prompts and RPC.

- Repository: earendil-works/pi
- GitHub: https://github.com/earendil-works/pi
- Human docs: https://grok-wiki.com/public/docs/earendil-works-pi-7860a70e44d1
- Complete Markdown: https://grok-wiki.com/public/docs/earendil-works-pi-7860a70e44d1/llms-full.txt

## Source Files

- `packages/coding-agent/test/agent-session-compaction.test.ts`
- `packages/coding-agent/test/suite/agent-session-compaction.test.ts`
- `packages/coding-agent/test/suite/regressions/5217-compaction-reason.test.ts`
- `packages/coding-agent/test/suite/regressions/6647-compaction-retries-transient-stream-drop.test.ts`
- `packages/coding-agent/test/suite/regressions/7253-manual-compact-during-response.test.ts`
- `packages/coding-agent/test/agent-session-auto-compaction-queue.test.ts`

---

---
title: "Context compaction"
description: "Auto and manual compaction triggers, reasons, retries on stream drop, and interactions with in-flight prompts and RPC."
---

`AgentSession` owns context compaction for every run mode (interactive, print/JSON, RPC, SDK). When history approaches the model context window, Pi summarizes older branch entries into a `compaction` session entry and rebuilds LLM context as **summary + kept recent messages**. Pure preparation and summarization live under `src/core/compaction/`; session I/O, reasons, retry/overflow policy, and prompt gating live in `src/core/agent-session.ts`.

## Compaction reasons

Every compaction run carries a `reason` on session events and extension hooks:

| Reason | Trigger | Auto-retry the interrupted turn? |
|--------|---------|----------------------------------|
| `manual` | `/compact [instructions]`, `session.compact()`, RPC `compact` | No (`willRetry: false`) |
| `threshold` | Context usage crosses the reserve-token floor after a turn (or on pre-prompt check) | No |
| `overflow` | Context overflow error, or recoverable `length` stop below the model’s desired output limit | Yes once when the assistant did not finish with `stopReason: "stop"` |

Extension events `session_before_compact` and `session_compact` report the same `reason` and `willRetry` values.

## Auto-compaction triggers

Auto-compaction runs only when `settings.compaction.enabled` is true (default). `AgentSession` checks the last assistant message:

1. After each agent run (`_handlePostAgentRun` → `_checkCompaction`)
2. Immediately before a new non-streaming `prompt()` when a prior assistant exists (covers aborted responses that left large context)

### Threshold

```
contextTokens > contextWindow - reserveTokens
```

Defaults: `reserveTokens = 16384`, `keepRecentTokens = 20000`. Threshold compaction does **not** call `agent.continue()` for a retry; the user (or a queued message) continues manually. If agent-level follow-up/steer messages are already queued, auto-compaction returns true so the post-run loop can continue delivery.

### Overflow recovery

Same-model failures classified as context overflow (`isContextOverflow`) or recoverable length truncation (`isRecoverableLength`) use reason `"overflow"`:

- Failed / truncated assistant (`stopReason` is not `"stop"`): remove the bad trailing assistant from agent state, compact once, set `willRetry: true`, then continue the turn.
- Successful assistant that finished over the window (`stopReason: "stop"`): compact with `willRetry: false` so a completed answer is not re-prompted via `agent.continue()`.
- At most **one** compact-and-retry attempt per recovery cycle (`_overflowRecoveryAttempted`). A second overflow after that emits:

  `Context overflow recovery failed after one compact-and-retry attempt. Try reducing context or switching to a larger-context model.`

Guards that skip auto-compaction:

| Guard | Effect |
|-------|--------|
| `compaction.enabled === false` | No auto path (manual still works) |
| Aborted assistant after agent_end | Skipped unless pre-prompt check (`skipAbortedCheck: false`) |
| Assistant from a different provider/model | No overflow recovery for the previous model’s error |
| Assistant timestamp ≤ latest compaction entry | Stale pre-compaction usage/error cannot re-fire |
| `prepareCompaction` returns undefined | Nothing to summarize / already compacted |

Context overflow is **not** treated as a normal provider auto-retry; `_isRetryableError` returns false when `isContextOverflow` matches so compaction owns recovery.

## Manual compaction

| Surface | Invocation |
|---------|------------|
| Interactive | `/compact` or `/compact <custom instructions>` |
| SDK / session API | `await session.compact(customInstructions?)` |
| RPC | `{"type":"compact"}` or with `customInstructions` |
| Toggle auto | `session.setAutoCompactionEnabled(enabled)` / RPC `set_auto_compaction` |

`compact()`:

1. Aborts the current agent operation (`await this.abort()`).
2. Sets `_compactionAbortController` and emits `compaction_start` with `reason: "manual"`.
3. Runs `prepareCompaction` on the current branch path.
4. Emits `session_before_compact` (extensions may cancel or supply a custom `CompactionResult`).
5. Otherwise calls `compact()` summarization with `settings.retry` and summarization retry callbacks.
6. Appends a `CompactionEntry` via `sessionManager.appendCompaction`, rebuilds `agent.state.messages`, emits `session_compact` and `compaction_end`.

Failure modes from preparation:

- `Already compacted` — leaf entry is already type `compaction`
- `Nothing to compact (session too small)` — no messages in the summarizable span
- `No model selected` / missing auth for summarization
- `Compaction cancelled` — extension cancel or `abortCompaction()`

Cancel any in-flight manual or auto compaction with `session.abortCompaction()` (aborts both abort controllers). Branch summarization uses `abortBranchSummary()` separately; both contribute to `isCompacting`.

## What the LLM sees after compaction

```
path entries (append-only tree)
  ... older messages ...
  compaction { summary, firstKeptEntryId, tokensBefore, details?, usage? }
  ... messages from firstKeptEntryId ...

LLM context rebuild
  system prompt
  + compactionSummary message (wrapped summary text)
  + messages from firstKeptEntryId onward
```

Preparation (`prepareCompaction`):

1. Finds the previous compaction boundary (iterative update uses its summary and `firstKeptEntryId`).
2. Estimates `tokensBefore` from the rebuilt session context.
3. Walks backward until `keepRecentTokens` is satisfied; cut points are user, assistant, bash, or custom/branch-summary messages — never tool results alone.
4. May produce a **split turn** when one turn exceeds the keep budget; history summary and turn-prefix summary are merged.
5. Tracks cumulative `readFiles` / `modifiedFiles` in `details` for default (non-extension) summaries.

Summarization requests use a fresh routing `sessionId` and `cacheRetention: "none"` so one-off summary prompts do not write reusable prompt cache entries.

## Summarization retries on stream drop

Compaction and branch-summary LLM calls go through `completeSummarization`, which wraps the single call in `retryAssistantCall` using **`settings.retry`** (same budget as turn auto-retry):

| Setting | Default |
|---------|---------|
| `retry.enabled` | `true` |
| `retry.maxRetries` | `3` |
| `retry.baseDelayMs` | `2000` (exponential backoff) |

Transient mid-stream failures (for example error message `terminated` / socket death) are retried. Deterministic non-retryable errors (for example `insufficient_quota`) and aborts fail immediately. Retry is skipped when `retry.enabled` is false.

Session events during summarization retry:

| Event | When |
|-------|------|
| `summarization_retry_scheduled` | Backoff scheduled: `attempt`, `maxAttempts`, `delayMs`, `errorMessage` |
| `summarization_retry_attempt_start` | Next attempt begins; `source: "compaction"` includes `reason` |
| `summarization_retry_finished` | Retry loop ends (success or exhaustion) |

`abortCompaction()` during backoff aborts the signal; the run ends as aborted (`compaction_end.aborted: true`).

## In-flight prompts and concurrent behavior

```mermaid
stateDiagram-v2
  [*] --> Idle
  Idle --> Streaming: prompt()
  Streaming --> Checking: agent_end
  Checking --> ThresholdCompact: shouldCompact
  Checking --> OverflowCompact: overflow / recoverable length
  Checking --> Idle: no compact
  OverflowCompact --> Streaming: willRetry continue
  ThresholdCompact --> Idle: no auto-retry
  Idle --> ManualCompact: compact() / /compact
  Streaming --> ManualCompact: compact() aborts agent first
  ManualCompact --> Idle: compaction_end
  note right of ManualCompact
    prompt() while manual compact
    throws until controller cleared
  end note
```

### Session API / SDK / RPC

While **manual** compaction holds `_compactionAbortController`, `prompt()` throws:

```text
Cannot submit a prompt while compaction is in progress. Wait for compaction to finish and retry.
```

The prompt is not persisted, does not start an agent run, and RPC `preflightResult` is false. Wait for `compaction_end` (or `isCompacting === false` from `get_state`) before retrying.

`compaction_end` clears the controller **before** listeners run so handlers may safely submit a queued prompt.

### Interactive TUI

While `session.isCompacting`:

- Normal user input is **queued** (`queueCompactionMessage`) as steer/follow-up for after compaction.
- Extension slash commands still run immediately via `session.prompt`.
- On `compaction_end`, `flushCompactionQueue` drains the queue; if `willRetry` is true, messages are steered/followed into the retry turn instead of starting a new top-level prompt.

### Manual compact during an active response

`compact()` aborts the agent, then runs only the **manual** compaction. A concurrent threshold that would have fired from the in-flight turn does not produce a second auto-compaction for that same turn—only the requested manual `compaction_start` / `compaction_end` with `reason: "manual"`.

### Auto-compaction and queues

Threshold auto-compaction with existing agent-level queued messages (follow-up/steer) signals continuation so those messages still deliver after rebuild. Overflow recovery with `willRetry: true` continues the interrupted turn after stripping a reappearing error/length assistant from rebuilt state.

## RPC surface

| Command | Role |
|---------|------|
| `compact` | Manual compaction; optional `customInstructions` |
| `set_auto_compaction` | Sets `compaction.enabled` via `setAutoCompactionEnabled` |
| `get_state` | Includes `isCompacting`, `autoCompactionEnabled` |

`compact` success payload matches `CompactionResult`:

| Field | Notes |
|-------|--------|
| `summary` | Summary text injected as context |
| `firstKeptEntryId` | First session entry id retained after the summary |
| `tokensBefore` | Pre-compaction context estimate |
| `estimatedTokensAfter` | Heuristic over rebuilt messages (not provider-exact) |
| `usage` | Optional LLM usage for summary generation |
| `details` | Default file ops or extension-defined data |

Streamed session events: `compaction_start`, `compaction_end`, `summarization_retry_*`. Overflow success with retry sets `willRetry: true` on `compaction_end`.

## Settings

Configure in `~/.pi/agent/settings.json` or project `.pi/settings.json`:

```json
{
  "compaction": {
    "enabled": true,
    "reserveTokens": 16384,
    "keepRecentTokens": 20000
  },
  "retry": {
    "enabled": true,
    "maxRetries": 3,
    "baseDelayMs": 2000
  }
}
```

| Key | Default | Effect |
|-----|---------|--------|
| `compaction.enabled` | `true` | Auto threshold/overflow only |
| `compaction.reserveTokens` | `16384` | Headroom for response; also summary max-token budget input |
| `compaction.keepRecentTokens` | `20000` | Recent tokens kept unsummarized |
| `retry.*` | see above | Summarization stream retries and turn auto-retry |

Interactive settings UI exposes auto-compact as **Auto-compact**. Disabling auto-compaction does not remove `/compact` or RPC `compact`.

## Extension hooks

| Hook | Can cancel | Can replace summary |
|------|------------|---------------------|
| `session_before_compact` | Yes (`{ cancel: true }`) | Yes (`{ compaction: CompactionResult }`) |
| `session_compact` | No | No (observe saved entry) |

`session_before_compact` receives `preparation` (`CompactionPreparation`), `branchEntries`, `customInstructions`, `reason`, `willRetry`, and `signal`. Extension-provided summaries skip the default LLM path but still append a session entry and rebuild context. See `examples/extensions/custom-compaction.ts` and `examples/extensions/trigger-compact.ts`.

## Failure modes

| Symptom | Likely cause |
|---------|----------------|
| Prompt / RPC error “compaction is in progress” | Manual compact still running; wait for `compaction_end` |
| `Already compacted` | Leaf is a compaction entry; send more turns before compacting again |
| `Nothing to compact (session too small)` | Keep budget covers entire path |
| Overflow recovery error after one attempt | Second overflow after compact-and-retry; reduce context or use a larger window model |
| Compaction fails with `terminated` then recovers | Expected when `retry.enabled`; watch `summarization_retry_*` |
| Compaction fails immediately with `terminated` | `retry.enabled: false` or non-retryable classification |
| Stale large usage after compact | Pre-compaction assistants are ignored by timestamp vs latest compaction entry |
| Auto never runs | `compaction.enabled: false` |

## Related pages

<CardGroup>
  <Card title="Agent sessions" href="/agent-sessions">
    Prompt queue, concurrent behavior, and turn ownership around compaction checks.
  </Card>
  <Card title="RPC mode" href="/rpc-mode">
    `compact`, `set_auto_compaction`, `isCompacting`, and prompt-during-compaction constraints.
  </Card>
  <Card title="Branching and session trees" href="/branching-and-tree">
    Branch summarization, tree navigation, and cancel-during-compact interactions.
  </Card>
  <Card title="Settings" href="/settings">
    Compaction and retry setting load/merge surfaces.
  </Card>
  <Card title="Extensions" href="/extensions">
    `session_before_compact` / `session_compact` registration and abort signals.
  </Card>
  <Card title="Troubleshooting" href="/troubleshooting">
    Retry, network, and settlement failures related to long sessions.
  </Card>
</CardGroup>