selfgraph First 30 Minutes Wiki

Complete Markdown

# selfgraph First 30 Minutes Wiki

> selfgraph is a minimal ActiveGraph agent that ingests its own source code, builds a capability graph from what it discovers, and proposes safe graph-native self-configuration patches validated by guardrails and tested in a forked sandbox before promotion to the live graph.

## Context Links

- [Agent index](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/llms.txt)
- [Human interactive wiki](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393)
- [GitHub repository](https://github.com/yoheinakajima/activegraph-selfgraph)

## Repository Metadata

- Repository: yoheinakajima/activegraph-selfgraph

- Generated: 2026-05-22T06:05:25.981Z
- Updated: 2026-05-22T06:49:13.181Z
- Runtime: Claude Code
- Format: First 30 Minutes
- Pages: 8

## Page Index

- 01. [Start Here: What selfgraph Is and How to Read It](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/01-start-here-what-selfgraph-is-and-how-to-read-it.md) - What this repo is, the mental model behind it, the key vocabulary (Capability, PatchProposal, guardrails, sandbox, promote), the fastest read order (README → cli.py → ingest/extract → propose → guardrails → sandbox), and the one constraint to keep in mind: the deterministic extractor is the contract, the LLM pass is optional and additive.
- 02. [Setup, CLI Commands & State Persistence](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/02-setup-cli-commands-state-persistence.md) - How to install (pip install -r requirements.txt, no API key required), the six CLI commands (build, ask, propose, promote, chat, demo), how state persists to .selfgraph/graph.db via Runtime.load/persist_to, and when to delete graph.db to force a cold rebuild.
- 03. [ingest.py & extract.py — Building the Capability Graph](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/03-ingest.py-extract.py-building-the-capability-graph.md) - How ingest.py walks the repo and introspects the activegraph module to produce File and Chunk objects (deduped on path + sha256), and how extract.py applies regex/heuristic patterns over those chunks to emit Capability, API, Behavior, ObjectType, Constraint, AuthorityRule, and RelationType nodes. Covers the deterministic vs. optional LLM-augment split and the SELFGRAPH_OBJECTTYPE_MATCH env flag (literal vs. relaxed) that controls which ObjectType regex fires.
- 04. [propose.py & query.py — Graph-Grounded Proposals and Answers](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/04-propose.py-query.py-graph-grounded-proposals-and-answers.md) - How propose_patch_for composes a PatchProposal from extracted Behaviors, EventTypes, and ObjectTypes already in the graph (and the [FALLBACK] scaffold path when no matching Behavior is found), and how answer_question uses keyword-overlap retrieval over node data — not semantic search — to answer questions. Covers the node and relation types emitted by a proposal (PatchProposal, Evaluation, Policy, BehaviorBinding, Task) and the GROUNDED_IN / PATCH_PROPOSES relations.
- 05. [guardrails.py — Validation Rules and PatchProposal Lifecycle](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/05-guardrails.py-validation-rules-and-patchproposal-lifecycle.md) - The allowed v1 change kinds (add_object, add_relation, add_policy, add_state_bucket, add_task, add_evaluation, bind_behavior), the substring banlist (_BANNED_TOKENS), the _PROTECTED_TYPES list blocking AuthorityRule/Capability mutation, and the draft → validated → applied (or rejected) state machine enforced at two call sites. Explains why cmd_promote re-runs validate_proposal with mutate_status=False before applying so a stale validated marker cannot bypass the check.
- 06. [sandbox.py — Fork, Diff, and Promote](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/06-sandbox.py-fork-diff-and-promote.md) - How sandbox_apply forks the SQLite-backed Runtime via Runtime.fork(at_event=...) or falls back to a structural replay on an in-memory graph, applies changes, emits a synthetic smoke TestEvent so newly bound behaviors fire, diffs added_objects and added_relations, and conditionally promotes to the live graph when promote=True. Covers the real-fork vs. in-memory fallback distinction and the single comment in sandbox.py marking where a public projector entry point would live.
- 07. [Test Suite & Reproducibility Harness](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/07-test-suite-reproducibility-harness.md) - What tests/test_smoke.py covers (accept path, banned-token injection, unknown-behavior binding, protected-type addition, disallowed change kind, promote lifecycle) and how the harness/ scripts (run_corpus.py, run_adversarial.py, run_future_event.py, extractor_recall.py, rollback_precondition.py, compare.py, report.py, invariants.py) regenerate the paper result files in harness/results/. Explains the LLM-free invariant enforced by the harness (ANTHROPIC_API_KEY must be unset) and the SELFGRAPH_OBJECTTYPE_MATCH=literal vs. relaxed condition that produces corpus.literal.jsonl vs. corpus.relaxed.jsonl.
- 08. [After 30 Minutes: What You Now Know and Where to Go Next](https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/08-after-30-minutes-what-you-now-know-and-where-to-go-next.md) - A closing map of what a reader should understand after this wiki — the full build→ask→propose→validate→sandbox→promote flow, the safety boundaries (no code authoring, no shell, no external I/O), and the key limitations to keep in mind (fallback scaffold, keyword-only retrieval, no multi-step planning, no UI). Suggests concrete next experiments: run demo.py, inspect graph.db with sqlite3, try a goal that triggers the FALLBACK branch, or add a new regex to extract.py and re-run the harness to verify the sha changes.

## Source File Index

- `harness/invariants.py`
- `harness/reproduce.sh`
- `harness/results/CANONICAL_SHAS.txt`
- `harness/run_corpus.py`
- `README.md`
- `REPRODUCE.md`
- `requirements.txt`
- `selfgraph/__init__.py`
- `selfgraph/__main__.py`
- `selfgraph/cli.py`
- `selfgraph/extract.py`
- `selfgraph/guardrails.py`
- `selfgraph/ingest.py`
- `selfgraph/propose.py`
- `selfgraph/query.py`
- `selfgraph/sandbox.py`
- `tests/test_harness.py`
- `tests/test_smoke.py`

---

## 01. Start Here: What selfgraph Is and How to Read It

> What this repo is, the mental model behind it, the key vocabulary (Capability, PatchProposal, guardrails, sandbox, promote), the fastest read order (README → cli.py → ingest/extract → propose → guardrails → sandbox), and the one constraint to keep in mind: the deterministic extractor is the contract, the LLM pass is optional and additive.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/01-start-here-what-selfgraph-is-and-how-to-read-it.md
- Generated: 2026-05-22T06:02:44.142Z

### Source Files

- `README.md`
- `selfgraph/__init__.py`
- `selfgraph/__main__.py`
- `requirements.txt`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [README.md](README.md)
- [selfgraph/__init__.py](selfgraph/__init__.py)
- [selfgraph/__main__.py](selfgraph/__main__.py)
- [selfgraph/cli.py](selfgraph/cli.py)
- [selfgraph/ingest.py](selfgraph/ingest.py)
- [selfgraph/extract.py](selfgraph/extract.py)
- [selfgraph/propose.py](selfgraph/propose.py)
- [selfgraph/guardrails.py](selfgraph/guardrails.py)
- [selfgraph/sandbox.py](selfgraph/sandbox.py)
- [requirements.txt](requirements.txt)
</details>

# Start Here: What selfgraph Is and How to Read It

selfgraph is a minimal ActiveGraph agent that ingests its own source repository and the ActiveGraph runtime, builds a **capability graph** from what it finds, and uses that graph to propose safe, graph-native self-configuration patches for new user goals. It is designed as a working demonstration of a self-modifying agent that is constrained by what it has actually discovered — if a primitive (a Behavior, an EventType, an ObjectType) is not in the graph, it does not appear in a proposal.

This page orients you for your first 30 minutes in the repo. It names the key terms, explains the mental model, lays out the recommended read order, and highlights the one architectural constraint that shapes everything else: **the deterministic extractor is the contract; the LLM pass is optional and purely additive**.

---

## The Mental Model

selfgraph turns a Python repository into a living, queryable graph and then uses that graph as both the knowledge base and the safety boundary for self-modification. There is no hidden prompt that tells the agent what it can do. Instead, the agent reads the graph, proposes changes that reference only nodes already in the graph, and is blocked from introducing anything — a behavior binding, a new ObjectType, a policy — that has not been extracted from real source code or documentation.

```text
┌─────────────────────────────────────────────────────────────────┐
│                     selfgraph pipeline                          │
│                                                                 │
│  [Repo / Package]                                               │
│       │                                                         │
│   ingest.py ──── File + Chunk objects ────► graph.db (SQLite)  │
│       │                                         │               │
│   extract.py ─── Capability, API, Behavior,     │               │
│                  ObjectType, Constraint, ◄──────┘               │
│                  AuthorityRule nodes                            │
│                        │                                        │
│   propose.py ─── PatchProposal (draft) ─────────────────────►  │
│                        │                                        │
│   guardrails.py ─ validate ─► validated | rejected              │
│                        │                                        │
│   sandbox.py ─── fork ─► apply ─► diff ─► (promote)            │
└─────────────────────────────────────────────────────────────────┘
```

Sources: [selfgraph/__init__.py:1-29](selfgraph/__init__.py), [README.md:76-90](README.md)

---

## Key Vocabulary

| Term | Where it lives | What it means |
|------|---------------|---------------|
| **Capability** | Graph node (`type="Capability"`) | A named thing the agent can do, seeded by `_SEED_CAPABILITIES` in `extract.py` and optionally augmented by the LLM pass. Six stable anchors are hardcoded (`ingest-repo`, `extract-capability`, `answer-question`, `propose-patch`, `validate-patch`, `sandbox-apply`). |
| **Behavior** | Graph node (`type="Behavior"`) | An `@behavior` / `@llm_behavior` / `@relation_behavior`-decorated Python function extracted from source. Identified by regex in `extract.py`. A `bind_behavior` change in a proposal must name a Behavior node that actually exists in the graph. |
| **PatchProposal** | Graph node (`type="PatchProposal"`) | The unit of self-modification. Carries `goal`, `rationale`, `changes` (a list of typed change dicts), `evaluation` criteria, and `status`. Created by `propose.py`, validated by `guardrails.py`, applied by `sandbox.py`. |
| **guardrails** | `selfgraph/guardrails.py` | Two-layer validation. Primary: only seven change `kind` values are allowed (`add_object`, `add_relation`, `add_policy`, `add_state_bucket`, `add_task`, `add_evaluation`, `bind_behavior`). Secondary: substring banlist over the whole proposal payload, plus a `_PROTECTED_TYPES` check that blocks adding `AuthorityRule` or `Capability` nodes without explicit approval. |
| **sandbox** | `selfgraph/sandbox.py` | A forked copy of the graph. When a SQLite-backed runtime is active, `Runtime.fork(at_event=...)` is used. Otherwise a structural replay into a fresh `Graph` object is used. Changes are applied to the fork first; only `promote=True` writes them to the live graph. |
| **promote** | `cmd_promote` in `cli.py` | The user-triggered step that re-validates the proposal (with `mutate_status=False`) against the *current* persisted graph state, then calls `sandbox_apply(..., promote=True)` to commit changes. A stale `validated` marker alone is never enough. |
| **AuthorityRule** | Graph node (`type="AuthorityRule"`) | Hardcoded rules in `extract.py` (`no-arbitrary-code`, `no-authority-mutation`, `no-external-side-effects`, `allowed-change-kinds`). Every Capability node carries a `CAPABILITY_REQUIRES_APPROVAL` edge to the `no-authority-mutation` rule. |
| **fallback scaffold** | `propose.py:84-138` | When no extracted Behavior matches the goal, the proposer falls back to a built-in atom/snapshot/`ROLLS_UP_INTO` structure. Proposals that took this path set `used_fallback_scaffold: true` and prefix the rationale with `[FALLBACK]`. |

Sources: [selfgraph/extract.py:96-128](selfgraph/extract.py), [selfgraph/guardrails.py:21-37](selfgraph/guardrails.py), [selfgraph/propose.py:84-104](selfgraph/propose.py), [selfgraph/cli.py:83-101](selfgraph/cli.py)

---

## PatchProposal Lifecycle

The status field on a `PatchProposal` node follows a strict transition path enforced at two call sites (not a state machine class):

```stateDiagram-v2
    [*] --> draft : propose_patch_for()
    draft --> validated : validate_proposal() ok=true
    draft --> rejected : validate_proposal() ok=false
    validated --> applied : sandbox_apply(promote=True)\nafter re-validation
    validated --> rejected : promote re-validation fails
```

- `validate_proposal` flips `draft → validated | rejected` and stamps the report onto the node.
- `cmd_promote` re-runs `validate_proposal(mutate_status=False)` — a pure check — before calling `sandbox_apply(..., promote=True)`.
- `sandbox_apply` refuses to act on any proposal not already in the `validated` state.

Sources: [README.md:136-144](README.md), [selfgraph/cli.py:83-101](selfgraph/cli.py), [selfgraph/sandbox.py:29-35](selfgraph/sandbox.py)

---

## Fastest Read Order

Reading the files in this order gives you the complete mental model in one pass:

### 1. `README.md` — Start here
The README is unusually accurate and self-critical. Read the *Architecture* section and *Limitations* in full. The limitations section (especially the `[FALLBACK]` note and the guardrails caveat) will save you from over-trusting the system's outputs.

### 2. `selfgraph/cli.py` — The entry points
All six commands (`build`, `ask`, `propose`, `promote`, `chat`, `demo`) are plain functions here, 10–20 lines each. Reading them in order is the fastest way to understand what each pipeline stage does and which modules it calls.

Sources: [selfgraph/cli.py:48-140](selfgraph/cli.py)

### 3. `selfgraph/ingest.py` — How source becomes graph nodes
`ingest_paths` walks the filesystem and emits `File` + `Chunk` objects (2000-char chunks, deduped on path + sha256). `ingest_module_docs` introspects a Python package — reflection over `inspect.getmembers` — and renders it as a synthetic `module://` corpus that the extractor reads like markdown.

Sources: [selfgraph/ingest.py:40-80](selfgraph/ingest.py), [selfgraph/ingest.py:125-170](selfgraph/ingest.py)

### 4. `selfgraph/extract.py` — How chunks become capability nodes
The deterministic pass runs first, always. It uses six regex patterns over chunk text: `@behavior` decorators, `@tool` registrations, `## def`/`## class` headings from synthetic module files, `ObjectType(name=...)` constructor calls, markdown code blocks (as Example nodes), and `must`/`must not` sentences (as Constraint nodes). The optional LLM pass runs after, on markdown chunks only, and adds `Capability` and `Constraint` nodes. It cannot remove or alter nodes from the deterministic pass.

Sources: [selfgraph/extract.py:36-91](selfgraph/extract.py), [selfgraph/extract.py:131-162](selfgraph/extract.py)

### 5. `selfgraph/propose.py` — How goals become patch proposals
`propose_patch_for` always does four things: (1) adds an `ObjectType` state bucket for the goal, (2) adds a `Task` object, (3) tries to bind existing `Behavior` nodes whose `on=` event lists overlap goal words — and falls back to the atom/snapshot scaffold if nothing matches, (4) adds a scoped `Policy` and `Evaluation` criteria. The resulting `PatchProposal` object is wired with `PATCH_PROPOSES` edges to the `Capability` nodes it uses.

Sources: [selfgraph/propose.py:31-215](selfgraph/propose.py)

### 6. `selfgraph/guardrails.py` + `selfgraph/sandbox.py` — Safety and apply
`validate_proposal` enforces the allowed change `kind` list, the banned-token substring scan, `_PROTECTED_TYPES`, and the `bind_behavior` known-behavior check. `sandbox_apply` builds a forked graph (SQLite fork preferred, structural replay fallback), applies the changes there, runs a smoke `TestEvent`, diffs against the original, and optionally promotes.

Sources: [selfgraph/guardrails.py:45-78](selfgraph/guardrails.py), [selfgraph/sandbox.py:16-73](selfgraph/sandbox.py)

---

## The One Constraint to Keep in Mind

> **The deterministic extractor is the contract; the LLM pass is optional and additive.**

This shapes every other design decision:

- Running `python -m selfgraph build .` without an `ANTHROPIC_API_KEY` produces the same graph shape on every machine. The paper harness actively refuses to run while `ANTHROPIC_API_KEY` is set unless `SELFGRAPH_HARNESS_ALLOW_LLM=1` is also set.
- When the LLM pass is enabled, it only calls the Anthropic API over markdown chunks, parses the JSON response, and adds `Capability` / `Constraint` nodes that are not already in the graph. It cannot remove or replace deterministically extracted nodes.
- The `SELFGRAPH_OBJECTTYPE_MATCH` environment variable (`literal` or `relaxed`, default `relaxed`) controls which regex patterns the deterministic pass uses to detect `ObjectType` names. This is how the paper's before/after A/B comparison is cold-reproducible without touching code.

Sources: [selfgraph/extract.py:63-91](selfgraph/extract.py), [selfgraph/extract.py:334-389](selfgraph/extract.py), [requirements.txt:1-8](requirements.txt), [README.md:27-30](README.md)

---

## Node Types and Relations at a Glance

**Node types** emitted by the pipeline:

| Stage | Node types created |
|-------|--------------------|
| `ingest.py` | `File`, `Chunk` |
| `extract.py` (deterministic seed) | `Capability`, `AuthorityRule` |
| `extract.py` (chunk scan) | `API`, `Behavior`, `EventType`, `ObjectType`, `Example`, `Constraint` |
| `extract.py` (LLM pass, optional) | `Capability`, `Constraint` |
| `propose.py` | `PatchProposal`, `Evaluation`, `Task` (via applied changes: `Policy`, `BehaviorBinding`) |

**Key relation types** wiring the graph:

| Relation | Source → Target |
|----------|----------------|
| `FILE_HAS_CHUNK` | File → Chunk |
| `BEHAVIOR_SUBSCRIBES_TO` | Behavior → EventType |
| `EXAMPLE_DEMONSTRATES` | Behavior → Chunk |
| `CAPABILITY_REQUIRES_APPROVAL` | Capability → AuthorityRule |
| `API_CREATES / API_READS / API_WRITES` | Capability → API |
| `PATCH_PROPOSES` | PatchProposal → Capability |
| `PATCH_MODIFIES` | PatchProposal → ObjectType |
| `GROUNDED_IN` | ObjectType (new atom) → ObjectType (existing) |
| `ROLLS_UP_INTO` | ObjectType (atom) → ObjectType (snapshot) |

Sources: [README.md:94-105](README.md), [selfgraph/extract.py:207-303](selfgraph/extract.py)

---

## Quick-Start Command Reference

```bash
# Install (Python 3.11+)
pip install -r requirements.txt          # activegraph==1.0.5.post2; anthropic optional

# Three-step scripted demo
python demo.py

# Or manually, state persists to .selfgraph/graph.db
python -m selfgraph build .              # ingest repo + activegraph package + extract
python -m selfgraph ask "what can you do?"
python -m selfgraph propose "track inbound emails from a vendor"
python -m selfgraph promote PatchProposal#NNN   # re-validates before applying
python -m selfgraph chat                 # interactive REPL
```

The state file `.selfgraph/graph.db` is a SQLite event log. Delete it to start fresh. `build` always wipes and recreates it (`create=True` in `_open`). All other commands call `Runtime.load(_DB_PATH)` to replay the event log and reconstruct the graph.

Sources: [selfgraph/cli.py:30-57](selfgraph/cli.py), [README.md:22-49](README.md)

---

## What selfgraph Intentionally Cannot Do

Understanding the hard limits saves debugging time:

- **No new Python functions.** `bind_behavior` can only reference behavior names already discovered by the extractor. The agent cannot author code.
- **No external side effects.** No HTTP, shell, or file writes outside the SQLite event store. `_BANNED_TOKENS` in `guardrails.py` blocks `subprocess`, `os.system`, `eval(`, `exec(`, `requests.`, `socket.`, and a dozen more.
- **No elevation of authority.** Patches cannot add `AuthorityRule` or `Capability` nodes without explicit `approved_by` argument. Policies cannot declare `can_approve`.
- **No multi-step planning.** One `PatchProposal` per goal; proposals are not chained.
- **No semantic question answering.** `query.py` is keyword-overlap retrieval over node data fields, not embedding search or LLM-based reasoning.

The README labels these as "v1, on purpose" limitations and explains the rationale for each. The safety guarantee rests on the structural constraint (only seven allowed change kinds, none of which can introduce code) rather than the substring banlist, which the README explicitly calls demo-grade and easy to evade on docs that merely *mention* banned tokens.

Sources: [README.md:159-194](README.md), [selfgraph/guardrails.py:21-37](selfgraph/guardrails.py)

---

## 02. Setup, CLI Commands & State Persistence

> How to install (pip install -r requirements.txt, no API key required), the six CLI commands (build, ask, propose, promote, chat, demo), how state persists to .selfgraph/graph.db via Runtime.load/persist_to, and when to delete graph.db to force a cold rebuild.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/02-setup-cli-commands-state-persistence.md
- Generated: 2026-05-22T06:02:21.825Z

### Source Files

- `requirements.txt`
- `selfgraph/cli.py`
- `selfgraph/__main__.py`
- `REPRODUCE.md`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [requirements.txt](requirements.txt)
- [selfgraph/cli.py](selfgraph/cli.py)
- [selfgraph/__main__.py](selfgraph/__main__.py)
- [selfgraph/sandbox.py](selfgraph/sandbox.py)
- [selfgraph/ingest.py](selfgraph/ingest.py)
- [selfgraph/query.py](selfgraph/query.py)
- [REPRODUCE.md](REPRODUCE.md)
</details>

# Setup, CLI Commands & State Persistence

This page explains how to install and run `activegraph-selfgraph`, describes each of its six CLI commands, and explains exactly how state is stored and reloaded between separate process invocations. It is the entry point for a developer's first 30 minutes in the repo — no prior knowledge of `activegraph` is assumed.

The pipeline is **LLM-free by design**: the canonical measured results require no API key. An optional LLM augmentation pass exists in `selfgraph/extract.py`, but the reproducibility harness actively refuses to run while `ANTHROPIC_API_KEY` is set, unless overridden with `SELFGRAPH_HARNESS_ALLOW_LLM=1`. Every result file carries an `llm_augment_active: false` audit stamp.

---

## Installation

### Requirements

```bash
# Python 3.11+ required
pip install -r requirements.txt
```

The only runtime dependency is `activegraph==1.0.5.post2`, pinned for paper reproducibility. The optional `anthropic` package is commented out in `requirements.txt`; do not install it unless you explicitly want the LLM augmentation variant (which produces different output hashes).

Sources: [requirements.txt:1-8]()

### Verifying the LLM-free invariant

You can confirm no LLM calls are wired into the core pipeline:

```bash
grep -nE 'anthropic|Anthropic|messages\.create|claude|llm_provider|LLMProvider' \
     selfgraph/propose.py selfgraph/guardrails.py selfgraph/sandbox.py \
     harness/*.py
# (no output expected)
```

Sources: [REPRODUCE.md:70-76]()

---

## Entry Point

`selfgraph/__main__.py` is a two-line shim that delegates entirely to `selfgraph/cli.py`:

```python
# selfgraph/__main__.py
from selfgraph.cli import main
import sys

if __name__ == "__main__":
    sys.exit(main())
```

All commands are invoked as:

```bash
python -m selfgraph <command> [args...]
```

Sources: [selfgraph/__main__.py:1-5](), [selfgraph/cli.py:1-12]()

---

## The Six CLI Commands

The command registry in `selfgraph/cli.py` maps six names to handler functions:

```python
_COMMANDS = {
    "build":   cmd_build,
    "ask":     cmd_ask,
    "propose": cmd_propose,
    "promote": cmd_promote,
    "chat":    cmd_chat,
    "demo":    cmd_demo,
}
```

Sources: [selfgraph/cli.py:118-125]()

### Command Summary Table

| Command | Requires existing graph? | Writes to graph.db? | Typical use |
|---------|--------------------------|---------------------|-------------|
| `build [repo_path]` | No (always cold-creates) | Yes | Initial or forced rebuild |
| `ask "question"` | Yes | No | Query the capability graph |
| `propose "goal"` | Yes | Yes (sandbox only) | Draft + validate a patch |
| `promote <proposal_id>` | Yes | Yes (live graph) | Apply a validated proposal |
| `chat` | Yes | No | Interactive question REPL |
| `demo` | No | Yes | Scripted end-to-end demo |

### `build` — Ingest and Extract

```bash
python -m selfgraph build [repo_path]
```

Wipes any existing `graph.db`, walks the repository at `repo_path` (defaults to `.`), and ingests the `activegraph` module docs. Then runs the capability extractor.

Under the hood `cmd_build` calls `_open(create=True)`, which unconditionally removes an existing database and creates a new `Graph` + `Runtime(graph, persist_to=_DB_PATH)` pair:

```python
def cmd_build(args: list[str]) -> int:
    repo = args[0] if args else "."
    graph, _rt = _open(create=True)
    ingest_paths(graph, [repo])
    ingest_module_docs(graph, "activegraph", max_submodules=40)
    extract_capabilities(graph)
    print(summarize_capabilities(graph))
    return 0
```

Sources: [selfgraph/cli.py:48-57]()

**What gets ingested:**
- Every text file (`.py`, `.md`, `.toml`, `.yaml`, `.json`, `.cfg`, `.ini`, `.rst`, `.txt`) under `repo_path`, excluding `.git`, `__pycache__`, `.venv`, `node_modules`, and files larger than 200 KB.
- Each file becomes a `File` object in the graph; files over 2000 characters are split into linked `Chunk` objects via `FILE_HAS_CHUNK` relations.
- The `activegraph` package is introspected live (up to 40 submodules), producing synthetic `module://name` file objects from docstrings and signatures.

Sources: [selfgraph/ingest.py:26-30](), [selfgraph/ingest.py:83-122](), [selfgraph/ingest.py:125-169]()

### `ask` — Query the Capability Graph

```bash
python -m selfgraph ask "what can you do?"
python -m selfgraph ask "how would you implement forking?"
python -m selfgraph ask "list constraints"
```

Loads the persisted graph and routes the question to one of several graph-readers in `selfgraph/query.py`. This is keyword-overlap retrieval, not LLM inference — every answer cites the node IDs it came from.

Routing logic:
- Questions starting with `"what can you do"` or containing `"capabilities"` → `summarize_capabilities()`
- Questions starting with `"how would you implement"` → `_explain_implementation()` (stem-matched graph walk)
- Questions starting with `"list "` or `"show "` → `_list_by_type()` (type filter over graph)
- Everything else → `_grep_graph()` (substring search across all object data)

Sources: [selfgraph/query.py:34-52]()

### `propose` — Draft and Validate a Patch

```bash
python -m selfgraph propose "track project updates"
```

Calls `propose_patch_for(graph, goal)` to generate a `PatchProposal` object, then immediately runs `validate_proposal` to check it against guardrails. If validation passes, `sandbox_apply(..., promote=False)` runs the proposal in an isolated fork and prints a diff summary. The proposal is not applied to the live graph.

```python
def cmd_propose(args: list[str]) -> int:
    goal = " ".join(args) or "track project updates"
    graph, rt = _open()
    pid = propose_patch_for(graph, goal)
    report = validate_proposal(graph, pid)
    if report["ok"]:
        sandbox = sandbox_apply(graph, pid, runtime=rt, promote=False)
        print(f"[propose] to promote: python -m selfgraph promote {pid}")
    return 0 if report["ok"] else 1
```

The command prints the `proposal_id` to copy-paste into `promote`.

Sources: [selfgraph/cli.py:67-80]()

### `promote` — Apply a Validated Proposal

```bash
python -m selfgraph promote <proposal_id>
```

Re-validates the proposal against the **current** persisted graph (the graph may have changed since `propose` ran), then calls `sandbox_apply(..., promote=True)` to write the changes to the live graph. The re-validation uses `mutate_status=False` to prevent overwriting the existing lifecycle status:

```python
report = validate_proposal(graph, pid, mutate_status=False)
if not report["ok"]:
    print(f"[promote] revalidation failed: {report['violations']}")
    return 1
sandbox_report = sandbox_apply(graph, pid, runtime=rt, promote=True)
```

A `PatchProposal` moves through the lifecycle: `draft → validated → applied`. The transition to `applied` is stamped in the event log under `actor="promote"`.

Sources: [selfgraph/cli.py:83-101](), [selfgraph/sandbox.py:63-69]()

### `chat` — Interactive REPL

```bash
python -m selfgraph chat
```

Opens an interactive prompt backed by `repl(graph)` from `selfgraph/query.py`. Every question typed is passed to `answer_question(graph, q)`. Type `quit`, `exit`, or `:q` to exit. No writes to the graph are made.

```
selfgraph chat — try: 'what can you do?', 'how would you implement forking?', 'list constraints', 'quit'
> what can you do?
...
```

Sources: [selfgraph/cli.py:104-108](), [selfgraph/query.py:326-339]()

### `demo` — Scripted End-to-End Run

```bash
python -m selfgraph demo
```

Imports `demo` and calls `demo.run()`, which executes a canned ingest → ask → propose → promote sequence. This is equivalent to running `demo.py` directly and is useful for a quick smoke-test of the full pipeline.

Sources: [selfgraph/cli.py:110-115]()

---

## State Persistence: How `graph.db` Works

### Storage Location

All state is stored in `.selfgraph/graph.db` relative to the working directory. The constants in `cli.py` define this:

```python
_DB_DIR  = ".selfgraph"
_DB_PATH = f"{_DB_DIR}/graph.db"
_RUN_ID  = "selfgraph"
```

The directory is created automatically on first use. `graph.db` is a **SQLite event store** — the activegraph `SQLiteEventStore` appends one event record per `add_object`, `add_relation`, `patch_object`, etc.

Sources: [selfgraph/cli.py:30-32]()

### The `_open()` Function

Every command goes through `_open()` to get a `(Graph, Runtime)` pair:

```python
def _open(create: bool = False) -> tuple[Graph, Runtime]:
    Path(_DB_DIR).mkdir(exist_ok=True)
    if create and Path(_DB_PATH).exists():
        os.remove(_DB_PATH)
    if create or not Path(_DB_PATH).exists():
        graph = Graph(ids=IDGen(), run_id=_RUN_ID)
        rt = Runtime(graph, persist_to=_DB_PATH)
        return graph, rt
    # Reuse existing log: load() rebuilds the graph from the event store.
    rt = Runtime.load(_DB_PATH, run_id=_RUN_ID)
    return rt.graph, rt
```

Two distinct paths:

| Path | Trigger | What happens |
|------|---------|--------------|
| **Cold create** | `create=True` or no `graph.db` exists | Deletes existing file, makes fresh `Graph`, wraps in `Runtime(graph, persist_to=_DB_PATH)`. Future mutations are appended to the new store. |
| **Warm load** | `graph.db` already exists, `create=False` | Calls `Runtime.load(_DB_PATH, run_id=_RUN_ID)` which replays the full event log into a fresh in-memory `Graph`. The resulting `rt.graph` reflects the complete current state. |

Sources: [selfgraph/cli.py:35-45]()

### The Append-Only Event Log

The SQLite file is not a snapshot — it is an ordered event log. When `Runtime.load` opens it, it replays every event through `Graph._replay_event` in insertion order to reconstruct the current graph state. This is the same mechanism used by `Runtime.fork` (for sandboxed proposals) and the rollback precondition test in the harness.

The rollback test demonstrates this property explicitly: replaying all events for a run up to (but not including) the first `promote`-actor event reconstructs a snapshot **byte-identical** to the pre-promote state. All 72 relaxed-corpus proposals verified this on the reference machine.

Sources: [selfgraph/sandbox.py:102-108](), [REPRODUCE.md:189-205]()

### The Sandbox Fork

When `propose` or `promote` runs, `sandbox_apply` creates an isolated copy of the graph using `Runtime.fork`:

```
graph.db (live, run_id="selfgraph")
    └─ Runtime.fork(at_event=last_event, label="selfgraph-sandbox")
           └─ fork_graph  (separate run_id, same SQLite file)
```

The fork shares the SQLite file but operates under a distinct `run_id`, so sandbox changes neither contaminate the main pipeline nor show up in the live graph unless `promote=True` is passed. If the runtime is not SQLite-backed, `sandbox_apply` falls back to replaying all events into a fresh in-memory `Graph`.

Sources: [selfgraph/sandbox.py:79-99]()

---

## Lifecycle of a PatchProposal

```text
propose "goal"
  └─ propose_patch_for()  → PatchProposal (status: "draft")
  └─ validate_proposal()  → status: "validated" (or "rejected")
  └─ sandbox_apply(promote=False) → fork diff preview

promote <proposal_id>
  └─ validate_proposal(mutate_status=False) → re-check against current graph
  └─ sandbox_apply(promote=True) → apply to live graph
       └─ patch_object(status: "applied", actor="promote")
```

Sources: [selfgraph/cli.py:67-101](), [REPRODUCE.md:259-267]()

---

## When to Delete `graph.db` (Forcing a Cold Rebuild)

Delete `.selfgraph/graph.db` whenever you need to:

1. **Start fresh** — e.g., after large changes to the repo that should be re-ingested from scratch.
2. **Run `build` against a different target directory** — `build` always calls `_open(create=True)` and removes the existing file automatically.
3. **Reproduce canonical harness results** — `harness/reproduce.sh` wipes persisted state before running the pipeline cold to ensure stable output hashes.
4. **Clear stale proposals** — proposals are stored in the graph; if you want to discard all previous `PatchProposal` objects, deleting and rebuilding is the simplest path (there is no `reset` command).

Running `python -m selfgraph build` is itself a safe delete-and-rebuild: the `_open(create=True)` path removes the file before starting. You do not need to delete it manually before a `build`.

Sources: [selfgraph/cli.py:37-38](), [REPRODUCE.md:10-14]()

---

## Data Flow Summary

```text
Working directory
  └─ .selfgraph/
       └─ graph.db   ← SQLite event store (append-only)

python -m selfgraph build .
  ┌──────────────────────────────────────────────┐
  │  ingest_paths(graph, ["."])                   │  → File + Chunk objects
  │  ingest_module_docs(graph, "activegraph")     │  → module:// File objects
  │  extract_capabilities(graph)                  │  → Capability, API, Behavior…
  └──────────────────────────────────────────────┘
        ↓ all mutations persisted via SQLiteEventStore

python -m selfgraph ask "…"
  └─ Runtime.load(graph.db)   ← replays event log into Graph
  └─ answer_question(graph, q) ← keyword lookup, no LLM

python -m selfgraph propose "…"
  └─ Runtime.load(graph.db)
  └─ propose_patch_for() + validate_proposal()
  └─ sandbox_apply(promote=False) ← fork, diff, no live write

python -m selfgraph promote <pid>
  └─ Runtime.load(graph.db)
  └─ sandbox_apply(promote=True) ← writes to live graph → event log
```

Sources: [selfgraph/cli.py:35-101](), [selfgraph/ingest.py:83-122]()

---

Every command reads from and writes to the same `.selfgraph/graph.db` file, which means `build`, `ask`, `propose`, and `promote` can safely run as separate processes — the event-store replay in `Runtime.load` ensures each process sees a consistent graph regardless of which process last wrote to the store. The single exception is `build`, which wipes and recreates the file, so it should not run concurrently with any other command.

Sources: [selfgraph/cli.py:11-45]()

---

## 03. ingest.py & extract.py — Building the Capability Graph

> How ingest.py walks the repo and introspects the activegraph module to produce File and Chunk objects (deduped on path + sha256), and how extract.py applies regex/heuristic patterns over those chunks to emit Capability, API, Behavior, ObjectType, Constraint, AuthorityRule, and RelationType nodes. Covers the deterministic vs. optional LLM-augment split and the SELFGRAPH_OBJECTTYPE_MATCH env flag (literal vs. relaxed) that controls which ObjectType regex fires.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/03-ingest.py-extract.py-building-the-capability-graph.md
- Generated: 2026-05-22T06:04:21.325Z

### Source Files

- `selfgraph/ingest.py`
- `selfgraph/extract.py`
- `selfgraph/cli.py`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [selfgraph/ingest.py](selfgraph/ingest.py)
- [selfgraph/extract.py](selfgraph/extract.py)
- [selfgraph/cli.py](selfgraph/cli.py)
</details>

# ingest.py & extract.py — Building the Capability Graph

`ingest.py` and `extract.py` are the two-stage pipeline that transforms a raw repository and Python module into a queryable capability graph. `ingest.py` walks file trees and introspects Python packages to produce `File` and `Chunk` objects, while `extract.py` applies deterministic regex and heuristic patterns over those chunks to emit higher-level graph nodes — `Capability`, `API`, `Behavior`, `ObjectType`, `Constraint`, `AuthorityRule`, and `RelationType`. Together they form the "build" step that every downstream query, proposal, and validation command depends on.

Understanding this pipeline matters because the graph is append-only and event-sourced: every node and edge added here is a durable, traceable fact. The doc comment in `ingest.py` states the design intent directly: "the trace is the proof the agent really read what it claims to know."

---

## Stage 1 — ingest.py: Files and Chunks

### Entry Points

The CLI's `cmd_build` in `cli.py` calls both entry points in sequence:

```python
# selfgraph/cli.py, lines 52-54
ingest_paths(graph, [repo])
ingest_module_docs(graph, "activegraph", max_submodules=40)
extract_capabilities(graph)
```

There are two ingestion paths:

| Function | Source | Emitted File kind |
|---|---|---|
| `ingest_paths` | Filesystem walk of one or more root paths | `"repo"` |
| `ingest_module_docs` | Live Python `importlib` introspection | `"module"` |

### ingest_paths — Filesystem Walk

`ingest_paths` accepts a list of root paths and walks each one with `os.walk`, skipping directories named `.git`, `__pycache__`, `.venv`, and `node_modules`. Only files whose suffix is in `TEXT_EXT` (`.md`, `.py`, `.toml`, `.yaml`, `.json`, etc.) are ingested, and files larger than 200 000 bytes are silently skipped.

A subtle but important detail is that `dirnames` and `filenames` are both sorted before processing, making the walk order stable across machines regardless of filesystem ordering:

```python
# selfgraph/ingest.py, lines 108-110
dirnames[:] = sorted(d for d in dirnames if d not in skip)
for fn in sorted(filenames):
    files.append(Path(dirpath) / fn)
```

Sources: [selfgraph/ingest.py:83-122]()

### ingest_module_docs — Module Introspection

`ingest_module_docs` imports a Python package with `importlib.import_module`, then uses `pkgutil.walk_packages` to enumerate every submodule. For each module it builds a synthetic Markdown-like text document using `_render_module`, which serialises:

- The module docstring
- Each public class (name, constructor signature, class docstring, public method signatures and docstrings)
- Each public function (name, signature, docstring)

The synthetic document is stored with a `module://` pseudo-path (e.g., `module://activegraph.graph`) so downstream extraction can identify it as a module artifact. Script-style entry points that call `sys.exit()` at import time are skipped to avoid crashing the ingestion process.

Sources: [selfgraph/ingest.py:125-170]()

### _emit_file — Deduplication on (path, sha256)

Both ingestion paths ultimately call `_emit_file`, which handles deduplication. Before creating a new `File` object the function checks whether an existing `File` already matches both the `path` and the `sha256` hash of the content:

```python
# selfgraph/ingest.py, lines 49-53
digest = _sha(content)
for existing in graph.objects(type="File"):
    if (existing.data.get("path") == path
            and existing.data.get("sha256") == digest):
        return existing.id
```

If an unchanged file is found, its id is returned and no new objects are emitted — re-running `build` is safe. If the content has changed, a new `File` is created (the old one is not deleted), so history is preserved.

After creating the `File`, content is split into `Chunk` objects of at most 2000 characters (`CHUNK_CHARS = 2000`). Each chunk is linked to its parent `File` via a `FILE_HAS_CHUNK` relation. Chunks carry their own `sha256` for later deduplication by the extractor.

Sources: [selfgraph/ingest.py:41-80]()

### File Object Schema

| Field | Value |
|---|---|
| `path` | Filesystem path or `module://` pseudo-path |
| `kind` | `"repo"` or `"module"` |
| `ext` | Lowercased file extension |
| `sha256` | First 16 hex digits of the content hash |
| `ingested_at` | UTC ISO-8601 timestamp |
| `size` | Content length in bytes |
| `preview` | First 400 characters of content |

---

## Stage 2 — extract.py: Capability Graph Nodes

`extract_capabilities` is the single entry point for this stage. It always runs the deterministic pass, and optionally runs an LLM augmentation pass when `ANTHROPIC_API_KEY` is set.

```
text
┌──────────────────────────────────────────────────────────────┐
│  extract_capabilities(graph)                                 │
│                                                              │
│  1. _seed()            → Capability + AuthorityRule anchors  │
│  2. _scan_chunk()      → API, Behavior, ObjectType,          │
│     (for each Chunk)     Constraint, RelationType, Example   │
│  3. _llm_augment()     → optional Capability + Constraint    │
│     (if API key set)     from doc chunks via Claude          │
└──────────────────────────────────────────────────────────────┘
```

### Seeding Stable Anchors

Before scanning any chunks, `_seed` writes six named `Capability` nodes and four `AuthorityRule` nodes into the graph. These are heuristic anchors that remain stable across re-runs; `_add_unique` prevents duplicates. Every `Capability` is then linked to the `no-authority-mutation` `AuthorityRule` via a `CAPABILITY_REQUIRES_APPROVAL` edge:

```python
# selfgraph/extract.py, lines 96-103
_SEED_CAPABILITIES = [
    ("ingest-repo",        "Read and chunk local files into File/Chunk objects."),
    ("extract-capability", "Mine signatures and docs for graph-native capabilities."),
    ("answer-question",    "Answer questions by querying the capability graph."),
    ("propose-patch",      "Generate a structured PatchProposal for a user goal."),
    ("validate-patch",     "Reject unsafe or out-of-scope patch proposals."),
    ("sandbox-apply",      "Apply a proposal in a fork, run test events, diff."),
]
```

Sources: [selfgraph/extract.py:96-128]()

### Deterministic Regex Patterns

The deterministic scan in `_scan_chunk` applies the following compiled patterns to each chunk's text:

| Pattern | Targets | Emits |
|---|---|---|
| `_RE_BEHAVIOR_DECO` | `@behavior`, `@llm_behavior`, `@relation_behavior` decorators | `Behavior` node + `BEHAVIOR_SUBSCRIBES_TO` edge for each `on=[...]` event |
| `_RE_TOOL_DECO` | `@tool(...)` decorator | `API` node with `kind="tool"` |
| `_RE_API_SIG` | `## def name(...)` / `## class name(...)` lines in `module://` files | `API` node; infers `API_CREATES`, `API_WRITES`, or `API_READS` relation from name |
| `_RE_OBJTYPE_HINT` + `_RE_OBJTYPE_CONSTRUCTOR` | `add_object("Type", ...)` and `ObjectType(name="...")` calls | `ObjectType` node |
| `` ``` `` in `.md` files | Markdown code fences | `Example` node |
| `_RE_MUST` | Sentences containing "must" or "must not" | `Constraint` node |

Every emitted object carries `source_chunk_id` and `source_file_path` metadata so grounding traces can cite back to the exact ingested artifact.

Sources: [selfgraph/extract.py:36-63](), [selfgraph/extract.py:207-305]()

### API Relation Inference

When processing `module://` synthetic files, the extractor uses a simple name-based heuristic to classify which relation an API node should have to its owning `Capability`. The name of the symbol is lowercased and matched against known vocabulary:

```python
# selfgraph/extract.py, lines 254-260
rel = (
    "API_CREATES" if any(k in lname for k in
                         ("add_", "create", "emit", "propose"))
    else "API_WRITES" if any(k in lname for k in
                             ("apply", "patch", "update", "remove"))
    else "API_READS"
)
```

This is a heuristic — all unrecognized names default to `API_READS`.

Sources: [selfgraph/extract.py:244-267]()

### The SELFGRAPH_OBJECTTYPE_MATCH Flag

Two regex patterns exist for ObjectType discovery, controlled by the `SELFGRAPH_OBJECTTYPE_MATCH` environment variable:

| Mode | Env Value | Regexes Active | What it catches |
|---|---|---|---|
| **Relaxed** (default) | `"relaxed"` or unset | `_RE_OBJTYPE_HINT` + `_RE_OBJTYPE_CONSTRUCTOR` | Capitalized `add_object("Type", ...)` calls **and** lowercase `ObjectType(name="company")` constructor calls used by the activegraph runtime |
| **Literal** | `"literal"` | `_RE_OBJTYPE_HINT` only | Only capitalized identifiers in `add_object(...)` / `ObjectType(name=...)` |

Any other value raises `ValueError` immediately — a deliberate guard so a typo does not silently produce shifted graph content:

```python
# selfgraph/extract.py, lines 83-91
mode = os.environ.get("SELFGRAPH_OBJECTTYPE_MATCH", "relaxed")
if mode == "literal":
    return [_RE_OBJTYPE_HINT]
if mode == "relaxed":
    return [_RE_OBJTYPE_HINT, _RE_OBJTYPE_CONSTRUCTOR]
raise ValueError(
    f"SELFGRAPH_OBJECTTYPE_MATCH={mode!r}; expected "
    f"'literal' or 'relaxed'"
)
```

The code comment explains the purpose: the activegraph runtime registers ObjectTypes with lowercase names like `"company"` and `"document"`, which the original literal regex misses entirely. The split between the two modes makes A/B reproducibility ("BEFORE/AFTER") cold-reproducible — set the env var to reproduce either baseline.

Sources: [selfgraph/extract.py:49-91]()

### Optional LLM Augmentation Pass

When `ANTHROPIC_API_KEY` is set, `_llm_augment` makes a second pass over the first eight `.md` chunks in the graph. It sends each chunk's first 1500 characters to `claude-sonnet-4-6` with a structured prompt asking for a JSON object containing `capabilities` (list of `{name, description}`) and `constraints` (list of strings). Returned nodes are merged via `_add_unique` so the LLM cannot create duplicates of deterministically extracted nodes.

The LLM pass is strictly additive: the deterministic pass always runs first and cannot be bypassed. If the Anthropic SDK is not installed or any exception occurs, the LLM pass is skipped silently and `counts["llm_added"]` is set to 0. LLM-sourced nodes carry `"source": "llm"` in their data to distinguish them from deterministic extractions.

Sources: [selfgraph/extract.py:334-389]()

### _add_unique — Deduplication in the Extract Pass

All nodes emitted by `_scan_chunk` go through `_add_unique`, which checks existing objects of the same type for a matching key before inserting. The key is derived from `name`, then `text`, then `snippet`:

```python
# selfgraph/extract.py, lines 308-319
def _add_unique(graph: Graph, type_: str, data: dict):
    key = data.get("name") or data.get("text") or data.get("snippet", "")
    if not key:
        return graph.add_object(type_, data, actor="extract")
    for o in graph.objects(type=type_):
        existing_key = o.data.get("name") or o.data.get("text") \
                       or o.data.get("snippet", "")
        if existing_key == key:
            return None
    return graph.add_object(type_, data, actor="extract")
```

This prevents the same `Behavior`, `API`, or `Constraint` from being recorded twice even when multiple chunks reference the same symbol.

---

## Data Flow Diagram

```text
  selfgraph build
        │
        ├─ ingest_paths(graph, [repo])
        │       │
        │       └─ per text file → _emit_file()
        │               ├─ dedup check (path + sha256)
        │               ├─ File object  (kind="repo")
        │               └─ Chunk objects (2000-char slices)
        │                       └─ FILE_HAS_CHUNK relation
        │
        ├─ ingest_module_docs(graph, "activegraph")
        │       │
        │       └─ per submodule → _render_module() → _emit_file()
        │               ├─ File object  (kind="module", path="module://...")
        │               └─ Chunk objects
        │
        └─ extract_capabilities(graph)
                │
                ├─ _seed()        → Capability + AuthorityRule nodes
                │                   CAPABILITY_REQUIRES_APPROVAL edges
                │
                ├─ _scan_chunk()  (for each Chunk)
                │   ├─ @behavior/* → Behavior + BEHAVIOR_SUBSCRIBES_TO
                │   ├─ @tool       → API (kind=tool)
                │   ├─ module:// sigs → API + API_CREATES/READS/WRITES
                │   ├─ ObjectType  → ObjectType (literal | relaxed)
                │   ├─ must/must not → Constraint
                │   └─ ``` in .md → Example
                │
                └─ _llm_augment() [optional, if ANTHROPIC_API_KEY set]
                        └─ Capability + Constraint (source="llm")
```

---

## Reading Order for New Contributors

1. **`selfgraph/ingest.py:41-80`** — `_emit_file`: understand the dedup contract before anything else; every other piece of ingest delegates here.
2. **`selfgraph/ingest.py:125-169`** — `ingest_module_docs`: see how Python module introspection generates the `module://` synthetic corpus that feeds API extraction.
3. **`selfgraph/extract.py:65-91`** — `_objecttype_regexes`: read the env-flag gate and understand why two regexes exist before looking at `_scan_chunk`.
4. **`selfgraph/extract.py:207-305`** — `_scan_chunk`: the full deterministic pattern sweep; maps directly to the node-type table above.
5. **`selfgraph/cli.py:48-57`** — `cmd_build`: the wiring that calls the above in order, and which resets the database with `create=True` on each full rebuild.

The deterministic pass is the contract; the LLM pass is enrichment. Any feature or test that relies on reproducible graph content should run with `ANTHROPIC_API_KEY` unset and `SELFGRAPH_OBJECTTYPE_MATCH=literal` or `=relaxed` pinned explicitly.

Sources: [selfgraph/extract.py:131-162]()

---

## 04. propose.py & query.py — Graph-Grounded Proposals and Answers

> How propose_patch_for composes a PatchProposal from extracted Behaviors, EventTypes, and ObjectTypes already in the graph (and the [FALLBACK] scaffold path when no matching Behavior is found), and how answer_question uses keyword-overlap retrieval over node data — not semantic search — to answer questions. Covers the node and relation types emitted by a proposal (PatchProposal, Evaluation, Policy, BehaviorBinding, Task) and the GROUNDED_IN / PATCH_PROPOSES relations.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/04-propose.py-query.py-graph-grounded-proposals-and-answers.md
- Generated: 2026-05-22T06:02:52.391Z

### Source Files

- `selfgraph/propose.py`
- `selfgraph/query.py`
- `selfgraph/cli.py`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [selfgraph/propose.py](selfgraph/propose.py)
- [selfgraph/query.py](selfgraph/query.py)
- [selfgraph/cli.py](selfgraph/cli.py)
- [selfgraph/guardrails.py](selfgraph/guardrails.py)
- [selfgraph/extract.py](selfgraph/extract.py)
</details>

# propose.py & query.py — Graph-Grounded Proposals and Answers

`propose.py` and `query.py` are the two output surfaces of the selfgraph agent. `propose.py` turns a free-text user goal into a structured `PatchProposal` object wired into the capability graph — it composes changes only from primitives already extracted (Behaviors, EventTypes, ObjectTypes) and falls back to a labelled scaffold when none match. `query.py` answers questions about the graph using keyword-overlap retrieval over node data, explicitly avoiding LLM re-prompting; every answer cites the node IDs it came from.

Together they form the read-then-act loop: `query.py` lets a user inspect what the graph knows, and `propose.py` generates a proposal that modifies that same graph using only graph-native operations — no file writes, no shell, no arbitrary code.

---

## propose.py — Building a PatchProposal

### Entry point: `propose_patch_for`

`propose_patch_for(graph, goal, *, proposed_by="selfgraph")` is the single public function. It always returns the string ID of the newly created `PatchProposal` object.

Sources: [selfgraph/propose.py:31-215]()

The function follows a fixed five-step composition sequence:

```text
Step 1  add_object      ObjectType   ← state bucket named from goal keywords
Step 2  add_task        Task         ← goal encoded as a graph object
Step 3  bind_behavior   ...          ← re-use existing Behaviors (happy path)
        OR [FALLBACK]   add_object + add_relation ← atom/snapshot scaffold
Step 4  add_policy      Policy       ← scope + allowed creates
Step 5  add_evaluation  Evaluation × 4  ← testable success criteria
```

### Step 1 — State bucket (ObjectType)

`_bucket_name(goal)` turns the goal string into a safe ObjectType identifier by title-casing the first three alphanumeric words longer than two characters.

```python
# selfgraph/propose.py:230-234
def _bucket_name(goal: str) -> str:
    words = [w for w in goal.replace("/", " ").split() if w.isalnum()]
    keep = [w.capitalize() for w in words[:3] if len(w) > 2]
    return "".join(keep) or "GoalBucket"
```

The result (e.g., `"TrackProject"` for goal `"track project updates"`) becomes the `scope` key for the Policy added in Step 4.

### Step 2 — Task object

A `Task` with `goal`, `bucket`, and `status: "pending"` is always added. This ensures the user's intent is itself a first-class graph node that downstream Behaviors can subscribe to.

Sources: [selfgraph/propose.py:56-67]()

### Step 3 — Behavior binding vs. fallback scaffold

`_pick_behavior_bindings(extracted, goal)` scans every `Behavior` in the graph, checks whether any goal token (words longer than 3 chars) overlaps the behavior's `name` or `on=` event types, and returns up to three `(behavior_name, event_type)` pairs.

```python
# selfgraph/propose.py:265-278
def _pick_behavior_bindings(extracted: dict, goal: str) -> list[tuple[str, str]]:
    goal_tokens = {t.lower() for t in goal.split() if len(t) > 3}
    out: list[tuple[str, str]] = []
    for b in extracted["behaviors"]:
        name = b.data.get("name", "")
        on_list = b.data.get("on") or []
        for ev in on_list:
            if any(tok in (name + " " + ev).lower() for tok in goal_tokens):
                out.append((name, ev))
                if len(out) >= 3:
                    return out
    return out
```

**Happy path:** each matching pair becomes a `bind_behavior` change, pointing to an already-extracted Behavior. The guardrail in `validate_proposal` will later verify the named Behavior exists in the graph.

**Fallback path (no matching Behavior):** when `bound` is empty, the proposer switches to the built-in atom/snapshot scaffold. This is explicitly labelled in both the `rationale` string and the `used_fallback_scaffold: true` flag on the proposal object. The scaffold path:

1. Calls `_dominant_event_type(extracted)` to pick the most-referenced EventType from ingested Behaviors as the trigger — observed from the graph, not hardcoded.
2. Adds two new ObjectTypes: `{Bucket}Atom` (individual records) and `{Bucket}Snapshot` (aggregated view).
3. Adds a `ROLLS_UP_INTO` relation from atom to snapshot.
4. Calls `_related_object_types(extracted, goal)` (keyword overlap scoring) to find up to two existing ObjectTypes to wire via `GROUNDED_IN` — so even the fallback structure is anchored to real extracted nodes when possible.

Sources: [selfgraph/propose.py:84-137]()

```python
# selfgraph/propose.py:92-103  (fallback label)
trigger = _dominant_event_type(extracted, default="object.created")
atom_type = f"{bucket}Atom"
snapshot_type = f"{bucket}Snapshot"
rationale_lines.append(
    f"[FALLBACK] No discovered Behavior matched the goal. "
    f"Falling back to the built-in atom/snapshot scaffold "
    ...
)
```

### Step 4 — Scoped Policy

The `add_policy` change derives its `can_create` list from the set of ObjectType names introduced by `add_object`, `add_state_bucket`, and `add_task` changes in the same proposal — not from a hardcoded whitelist.

```python
# selfgraph/propose.py:141-145
creatable = sorted({
    c.get("type") or c.get("data", {}).get("name")
    for c in changes
    if c.get("kind") in ("add_object", "add_state_bucket", "add_task")
} - {None})
```

The policy always marks `AuthorityRule` as requiring approval, consistent with the `no-authority-mutation` rule seeded by `extract.py`.

Sources: [selfgraph/propose.py:139-160]()

### Step 5 — Evaluation criteria

Four testable success criteria are added as `Evaluation` objects:

| Criterion | What it checks |
|---|---|
| `A {bucket} object exists after apply` | State bucket was materialized |
| `At least one Task with goal='{goal}' exists` | Goal was encoded in graph |
| `No PatchProposal with status='rejected' was produced` | Apply was clean |
| `AuthorityRule objects are unchanged` | No authority mutation occurred |

Sources: [selfgraph/propose.py:163-174]()

### Graph materialization and edge wiring

After building the `changes` list, `propose_patch_for` calls `graph.add_object("PatchProposal", {...})` to write the proposal as a first-class Object. Two sets of edges are then added:

| Relation | From | To | Meaning |
|---|---|---|---|
| `PATCH_PROPOSES` | PatchProposal | Capability | Capabilities this proposal exercises |
| `PATCH_MODIFIES` | PatchProposal | ObjectType | Extracted types the proposal grounds itself in (via GROUNDED_IN changes) |

Sources: [selfgraph/propose.py:198-215]()

```python
# selfgraph/propose.py:200-213
for cap in graph.objects(type="Capability"):
    if cap.data.get("name") in {"propose-patch", "extract-capability"}:
        graph.add_relation(proposal.id, cap.id, "PATCH_PROPOSES", actor=proposed_by)
grounded_targets = {
    c.get("to_name") for c in changes
    if c.get("kind") == "add_relation"
    and c.get("rel_type") == "GROUNDED_IN"
    and c.get("to_type") == "ObjectType"
}
for ot in graph.objects(type="ObjectType"):
    if ot.data.get("name") in grounded_targets:
        graph.add_relation(proposal.id, ot.id, "PATCH_MODIFIES", actor=proposed_by)
```

### Node and relation types emitted by a proposal

```text
Objects added to the graph:
  PatchProposal   — the proposal itself; data.changes holds the full change list
  Task            — encodes the user goal as a graph node
  ObjectType(s)   — state bucket, and atom/snapshot types in fallback path
  Evaluation(s)   — testable success criteria (4 per proposal)
  Policy          — scoped permission boundary for the new bucket

Relations added:
  PATCH_PROPOSES   PatchProposal → Capability
  PATCH_MODIFIES   PatchProposal → ObjectType
  GROUNDED_IN      {Bucket}Atom  → existing ObjectType   (fallback path only)
  ROLLS_UP_INTO    {Bucket}Atom  → {Bucket}Snapshot       (fallback path only)
```

### `used_fallback_scaffold` flag

The boolean `used_fallback_scaffold` on the proposal's `data` dict is the machine-readable signal downstream readers use to surface when the structure came from the default scaffold rather than from discovered Behaviors. `trace_grounding` in `query.py` reads this flag and reports it in the citation header.

Sources: [selfgraph/propose.py:180-192](), [selfgraph/query.py:145-149]()

---

## query.py — Keyword-Overlap Retrieval

### Design philosophy

`query.py`'s module docstring states the intent precisely: this is keyword-overlap retrieval over the capability graph — not semantic understanding. Every answer cites node IDs; if nothing matches, the function says so rather than inventing an answer.

Sources: [selfgraph/query.py:1-9]()

### `answer_question` — routing logic

`answer_question(graph, question)` is the public dispatcher. It pattern-matches the lowercased question against five branches:

```python
# selfgraph/query.py:34-52
def answer_question(graph: Graph, question: str) -> str:
    q = question.strip().lower()
    if q.startswith("what can you do") or "capabilities" in q:
        return summarize_capabilities(graph)
    if q.startswith("how would you implement") or q.startswith("how would you "):
        topic = question.split(maxsplit=4)[-1] ...
        return _explain_implementation(graph, topic)
    if q.startswith("can you configure yourself") or "configure yourself" in q:
        return "(hardcoded guidance string)"
    if q.startswith("list ") or q.startswith("show "):
        return _list_by_type(graph, question)
    return _grep_graph(graph, question)
```

| Branch | Trigger | Implementation |
|---|---|---|
| `summarize_capabilities` | "what can you do", "capabilities" | Sorted list of Capability nodes with API edge count |
| `_explain_implementation` | "how would you implement" | Top-3 Capabilities, top-5 APIs/Behaviors by keyword hits |
| configure-yourself | "can you configure yourself" | Hardcoded guidance string pointing to `propose_patch_for` |
| `_list_by_type` | "list X", "show X" | Enumerate nodes by type name, up to 25 |
| `_grep_graph` | fallback | Substring search across all node data |

### `_explain_implementation` — the keyword-overlap engine

The core retrieval function converts the topic into tokens, applies crude stemming (strips `s`, `ing`, `ed` suffixes), then scores each node by counting stem hits against the full concatenation of its data values.

```python
# selfgraph/query.py:55-76
def _explain_implementation(graph: Graph, topic: str) -> str:
    tokens = [t for t in cleaned.lower().split() if len(t) > 2]
    stems = {t.rstrip("s").rstrip("ing").rstrip("ed") for t in tokens} | set(tokens)

    def hits(o):
        text = " ".join(str(v) for v in o.data.values()).lower()
        return sum(1 for s in stems if s in text)

    relevant_caps = sorted(graph.objects(type="Capability"), key=hits, reverse=True)[:3]
    relevant_apis = sorted(graph.objects(type="API"), key=hits, reverse=True)[:5]
    relevant_behaviors = sorted(graph.objects(type="Behavior"), key=hits, reverse=True)[:5]
    constraints = [c for c in graph.objects(type="Constraint") if hits(c)]
```

Results are shown only when `hits > 0`; node IDs are always included in the output so the answer is graph-cited. When no node overlaps the topic, the function explicitly says so rather than fabricating an answer.

Sources: [selfgraph/query.py:55-106]()

### `_list_by_type` — node type enumeration

`_list_by_type` checks which of a fixed candidate-type list (`Capability`, `API`, `Behavior`, `ObjectType`, `RelationType`, `Example`, `Constraint`, `AuthorityRule`, `PatchProposal`, `Evaluation`, `File`, `EventType`) appear in the question, then returns up to 25 objects per requested type.

Sources: [selfgraph/query.py:109-129]()

### `_grep_graph` — substring fallback

For unrecognised questions, `_grep_graph` does a case-insensitive substring search over the concatenated data values of every object in the graph, returning up to 20 matches. This is the last-resort path; it is not semantic search.

Sources: [selfgraph/query.py:304-320]()

### `trace_grounding` — citation walker

`trace_grounding(graph, proposal_id)` is a separate reader that walks `PATCH_PROPOSES`, `PATCH_MODIFIES`, and per-change `GROUNDED_IN` edges from a proposal and renders a full citation chain. Each extracted node is traced back to its `source_file_path` (set by `extract.py`'s `_scan_chunk`). Scaffold objects carry `source: selfgraph-fallback-scaffold` and are labelled `"[scaffold: built-in fallback shape, not extracted]"` rather than a real file path.

The per-change classification is delegated to `classify_change`, which returns one of four categories:

| Category | When |
|---|---|
| `grounded-in-extracted` | GROUNDED_IN targets an existing ObjectType, or bind_behavior targets a known Behavior |
| `built-in-scaffold` | `add_object` with `source=selfgraph-fallback-scaffold` |
| `self-authored` | `add_task`, `add_evaluation`, `add_policy`, `add_state_bucket` |
| `domain-new` | Any other `add_object` or `add_relation` introducing new state |

Sources: [selfgraph/query.py:132-301]()

---

## Full data-flow diagram

```mermaid
sequenceDiagram
    participant CLI as cli.py cmd_propose
    participant P as propose.py
    participant G as Graph (activegraph)
    participant GR as guardrails.py
    participant Q as query.py trace_grounding

    CLI->>P: propose_patch_for(graph, goal)
    P->>G: graph.objects(type="Behavior/EventType/ObjectType")
    note over P: _scan_self → extracted dict

    alt Behaviors match goal keywords
        P->>P: _pick_behavior_bindings → bind_behavior changes
    else No match
        P->>P: _dominant_event_type → trigger
        P->>P: _related_object_types → GROUNDED_IN targets
        P->>P: atom/snapshot scaffold changes
    end

    P->>G: graph.add_object("PatchProposal", {...})
    P->>G: graph.add_relation(proposal, capability, "PATCH_PROPOSES")
    P->>G: graph.add_relation(proposal, objecttype, "PATCH_MODIFIES")
    P-->>CLI: proposal_id

    CLI->>GR: validate_proposal(graph, proposal_id)
    GR-->>CLI: report {ok, violations}

    CLI->>Q: trace_grounding(graph, proposal_id)
    Q->>G: relations(source=proposal_id) PATCH_PROPOSES / PATCH_MODIFIES
    Q->>Q: classify_change per change entry
    Q-->>CLI: citation chain text
```

---

## Guardrails interaction

After `propose_patch_for` returns, `cli.py cmd_propose` immediately calls `validate_proposal`. The guardrail checks every change in `proposal.data["changes"]`:

- `kind` must be in `ALLOWED_KINDS` (`add_object`, `add_relation`, `add_policy`, `add_state_bucket`, `add_task`, `add_evaluation`, `bind_behavior`)
- No `subprocess`, `eval`, `exec`, or network tokens in any string value
- `add_object` targeting `AuthorityRule` or `Capability` requires `approved_by`
- `bind_behavior` names must exist in the graph — unknown behavior names are rejected

A `bind_behavior` referencing a Behavior that was never extracted will therefore fail validation, which means the happy-path binding route is only valid when the named Behavior is actually in the graph. This is intentional: it prevents the proposer from inventing capability names.

Sources: [selfgraph/guardrails.py:21-126]()

---

## CLI commands

| Command | Function | What it does |
|---|---|---|
| `python -m selfgraph ask "<question>"` | `cmd_ask` → `answer_question` | Queries the graph |
| `python -m selfgraph propose "<goal>"` | `cmd_propose` → `propose_patch_for` | Generates, validates, and sandbox-applies a proposal |
| `python -m selfgraph chat` | `cmd_chat` → `repl` | Interactive REPL loop over `answer_question` |

Sources: [selfgraph/cli.py:60-80](), [selfgraph/cli.py:104-107]()

---

## What to read first

1. **`selfgraph/propose.py:31-215`** — the single `propose_patch_for` function is self-contained and its comment blocks explain every design decision inline.
2. **`selfgraph/query.py:34-52`** — `answer_question` is four readable `if` branches; understanding the routing takes two minutes.
3. **`selfgraph/query.py:220-261`** — `classify_change` defines the four change categories that `trace_grounding` renders; knowing these categories makes grounding traces immediately readable.
4. **`selfgraph/guardrails.py:21-32`** — `ALLOWED_KINDS` and `_BANNED_TOKENS` define the entire v1 safety surface in 15 lines.

The key invariant tying these modules together: every node emitted by `propose.py` either cites an extracted graph node (and carries a `source_file_path` traceable to an ingested file) or carries `source: selfgraph-fallback-scaffold`, and `query.py`'s `trace_grounding` makes that distinction explicit for every change in the proposal. Sources: [selfgraph/query.py:188-199]()

---

## 05. guardrails.py — Validation Rules and PatchProposal Lifecycle

> The allowed v1 change kinds (add_object, add_relation, add_policy, add_state_bucket, add_task, add_evaluation, bind_behavior), the substring banlist (_BANNED_TOKENS), the _PROTECTED_TYPES list blocking AuthorityRule/Capability mutation, and the draft → validated → applied (or rejected) state machine enforced at two call sites. Explains why cmd_promote re-runs validate_proposal with mutate_status=False before applying so a stale validated marker cannot bypass the check.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/05-guardrails.py-validation-rules-and-patchproposal-lifecycle.md
- Generated: 2026-05-22T06:02:04.635Z

### Source Files

- `selfgraph/guardrails.py`
- `selfgraph/cli.py`
- `tests/test_smoke.py`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [selfgraph/guardrails.py](selfgraph/guardrails.py)
- [selfgraph/cli.py](selfgraph/cli.py)
- [selfgraph/propose.py](selfgraph/propose.py)
- [selfgraph/sandbox.py](selfgraph/sandbox.py)
- [tests/test_smoke.py](tests/test_smoke.py)
</details>

# guardrails.py — Validation Rules and PatchProposal Lifecycle

`selfgraph/guardrails.py` is the security boundary between the LLM-driven proposer and the live ActiveGraph event store. Every `PatchProposal` must pass through it before any graph mutation is allowed. It defines what changes the system can make at all (the v1 allowed-kind allowlist), what strings are unconditionally forbidden (the banned-token scan), which object types are off-limits without explicit human approval (the protected-types list), and the state machine—`draft → validated → applied` (or `rejected`)—that prevents a stale approval token from bypassing a re-check at promote time.

Understanding guardrails is essential before touching `propose.py`, `sandbox.py`, or the CLI, because every other subsystem defers to this module's verdict before writing anything to the persistent SQLite event store.

---

## Allowed v1 Change Kinds

The validator maintains a closed allowlist of the only `kind` values a change dict may carry. Any value outside this set is immediately rejected with a `disallowed-kind` violation.

```python
# selfgraph/guardrails.py:21-24
ALLOWED_KINDS = {
    "add_object", "add_relation", "add_policy", "add_state_bucket",
    "add_task", "add_evaluation", "bind_behavior",
}
```

| Kind | Effect when applied | Notes |
|---|---|---|
| `add_object` | Creates a new typed node | Blocked for `AuthorityRule`/`Capability` without approval |
| `add_relation` | Creates a directed edge between two existing nodes | Resolved by `(type, name)` index in `sandbox.py` |
| `add_policy` | Creates a `Policy` node scoped to an object type | `can_approve` key is also blocked (permission escalation) |
| `add_state_bucket` | Convenience alias for an `ObjectType`-class object | Treated as `add_object` internally in `sandbox_apply` |
| `add_task` | Creates a `Task` node with lifecycle fields | Same path as `add_object` in `_apply_changes` |
| `add_evaluation` | Creates an `Evaluation` node for acceptance criteria | Used by `propose_patch_for` to record success criteria |
| `bind_behavior` | Creates a `BehaviorBinding` linking an existing behavior to a scoped event | Behavior name must already exist in the graph |

Sources: [selfgraph/guardrails.py:21-24](), [selfgraph/sandbox.py:114-155]()

---

## The Banned-Token Scan (`_BANNED_TOKENS`)

Before per-change checks run, the validator performs a full-payload substring scan. The entire proposal data blob—including nested dicts and lists—is walked recursively, and any occurrence of any token in `_BANNED_TOKENS` fires a `banned-token` violation with a dotted path identifying where the match was found.

```python
# selfgraph/guardrails.py:27-32
_BANNED_TOKENS = (
    "subprocess", "os.system", "__import__", "exec(", "eval(",
    "shutil.rmtree", "open(", "urllib", "requests.", "socket.",
    "rm -rf", "curl ", "wget ", "/bin/sh", "/bin/bash", "popen",
    "compile(", "globals()", "setattr",
)
```

The recursive walker (`_scan_banned`) handles `str`, `dict`, and `list`/`tuple` values, building a dotted path string like `.changes[2].data.recipe` so violation reports name exactly where the hit occurred.

```python
# selfgraph/guardrails.py:129-140
def _scan_banned(payload, _path: str = "") -> Iterable[str]:
    if isinstance(payload, str):
        low = payload.lower()
        for tok in _BANNED_TOKENS:
            if tok in low:
                yield f"{_path}: {tok}"
    elif isinstance(payload, dict):
        for k, v in payload.items():
            yield from _scan_banned(v, f"{_path}.{k}")
    elif isinstance(payload, (list, tuple)):
        for i, v in enumerate(payload):
            yield from _scan_banned(v, f"{_path}[{i}]")
```

The module docstring explicitly flags this as "demo-grade substring matching"—it is not a sandbox or AST parser and can be bypassed with encoding tricks. The README carries the same caveat. The design intent is to catch straightforward injection attempts in a prototype, not to provide production-hardened sandboxing.

The test suite exercises the rejection path by injecting `"subprocess.Popen(['rm', '-rf', '/'])"` into a change's `data.recipe` field and asserting `banned-token` appears in `report["violations"]`.

Sources: [selfgraph/guardrails.py:27-32](), [selfgraph/guardrails.py:129-140](), [tests/test_smoke.py:53-68]()

---

## Protected Types (`_PROTECTED_TYPES`)

```python
# selfgraph/guardrails.py:37
_PROTECTED_TYPES = {"AuthorityRule", "Capability"}
```

`AuthorityRule` and `Capability` are the nodes that define what the agent is allowed to do. A proposal that adds one of these without an explicit human approval string fires a `protected-type` violation. The check only applies to `add_object` changes (you cannot add these types at all in v1 without approval; mutating existing ones is blocked implicitly because no mutation kind exists in `ALLOWED_KINDS`).

```python
# selfgraph/guardrails.py:101-107
if kind == "add_object":
    t = change.get("type")
    if t in _PROTECTED_TYPES and not approved_by:
        report["violations"].append(
            ("protected-type", i,
             f"cannot add {t} without explicit approval")
        )
```

The `approved_by` argument to `validate_proposal` is the human-approval bypass. It defaults to `None`; passing a non-empty string (e.g., a human reviewer's identifier) lifts the block. Neither `cmd_propose` nor `cmd_promote` in `cli.py` currently pass `approved_by`, so both call sites treat all protected-type additions as violations.

The smoke test `test_proposal_rejected_for_protected_type_add` injects both an `AuthorityRule` and a `Capability` change and asserts at least two `protected-type` violations appear in the report.

Sources: [selfgraph/guardrails.py:37](), [selfgraph/guardrails.py:101-107](), [tests/test_smoke.py:89-109]()

---

## Additional Per-Kind Rules

Beyond kind allowlisting and protected types, two more per-change rules fire:

### Permission Escalation (`add_policy` with `can_approve`)

A `Policy` change that declares a `can_approve` key is blocked with a `permission-escalation` violation. This prevents a proposal from granting itself the ability to approve its own future patches.

```python
# selfgraph/guardrails.py:108-113
if kind == "add_policy":
    policy = change.get("policy", {})
    if "can_approve" in policy:
        report["violations"].append(
            ("permission-escalation", i,
             "policies may not declare can_approve")
        )
```

### Unknown Behavior (`bind_behavior`)

A `bind_behavior` change must name a `Behavior` node that already exists in the graph. The validator queries the live graph for all `Behavior` objects and compares against the set of their `name` fields.

```python
# selfgraph/guardrails.py:115-123
if kind == "bind_behavior":
    beh_name = change.get("behavior")
    known = {b.data.get("name") for b in graph.objects(type="Behavior")}
    if beh_name not in known:
        report["violations"].append(
            ("unknown-behavior", i,
             f"behavior {beh_name!r} not in capability graph; "
             f"v1 only binds existing behaviors")
        )
```

This enforces the proposer's documented design principle: "Bind existing behaviors instead of inventing new ones." (`selfgraph/propose.py:69`)

Sources: [selfgraph/guardrails.py:108-123](), [selfgraph/propose.py:69]()

---

## Violation Report Shape

`validate_proposal` returns a dict with the following structure, regardless of `mutate_status`:

```python
{
    "checked": int,           # number of changes examined
    "violations": [           # list of 3-tuples
        (rule_name, change_index, detail_string),
        ...
    ],
    "ok": bool,               # True iff violations is empty
}
```

`change_index` is `-1` for the banned-token scan (which operates on the whole payload, not a specific change), and `0`-based for per-change checks. The `rule_name` string is one of: `banned-token`, `malformed-change`, `disallowed-kind`, `protected-type`, `permission-escalation`, `unknown-behavior`.

Sources: [selfgraph/guardrails.py:81-126]()

---

## The PatchProposal State Machine

A `PatchProposal` is a first-class `Object` in the ActiveGraph event store. Its `status` field drives the lifecycle enforced across three modules.

```stateDiagram-v2
    [*] --> draft : propose_patch_for() creates proposal
    draft --> validated : validate_proposal(mutate_status=True) → ok
    draft --> rejected : validate_proposal(mutate_status=True) → violations
    validated --> applied : sandbox_apply(promote=True)
    validated --> validated : validate_proposal(mutate_status=False) re-check (cmd_promote)
    rejected --> [*]
    applied --> [*]
```

### State Transitions in Detail

| From | To | Trigger | Location |
|---|---|---|---|
| (new) | `draft` | `propose_patch_for()` creates the object | `selfgraph/propose.py:180-193` |
| `draft` | `validated` | `validate_proposal()` with `mutate_status=True`, no violations | `selfgraph/guardrails.py:61-74` |
| `draft` | `rejected` | `validate_proposal()` with `mutate_status=True`, violations found | `selfgraph/guardrails.py:61-74` |
| `validated` | `applied` | `sandbox_apply(promote=True)` after passing re-check | `selfgraph/sandbox.py:63-69` |

Sources: [selfgraph/propose.py:180-193](), [selfgraph/guardrails.py:61-74](), [selfgraph/sandbox.py:63-69]()

---

## Two Call Sites: `cmd_propose` and `cmd_promote`

### `cmd_propose` — First validation (with status mutation)

```python
# selfgraph/cli.py:70-80
def cmd_propose(args: list[str]) -> int:
    goal = " ".join(args) or "track project updates"
    graph, rt = _open()
    pid = propose_patch_for(graph, goal)
    report = validate_proposal(graph, pid)          # mutate_status=True (default)
    ...
    if report["ok"]:
        sandbox = sandbox_apply(graph, pid, runtime=rt, promote=False)
```

At propose time, `validate_proposal` is called with the default `mutate_status=True`. A passing proposal is stamped `validated`; a failing one is stamped `rejected`. The sandbox is then run with `promote=False`—changes are applied in an isolated fork so the caller can preview the diff without touching the live graph.

### `cmd_promote` — Re-validation without status mutation

```python
# selfgraph/cli.py:83-101
def cmd_promote(args: list[str]) -> int:
    ...
    # Re-validate against the current persisted state — the graph may
    # have changed between propose and promote (new ingestions, other
    # patches), so a stale 'validated' marker is not enough.
    # mutate_status=False so a re-check doesn't overwrite the existing
    # lifecycle status on the proposal.
    report = validate_proposal(graph, pid, mutate_status=False)
    if not report["ok"]:
        print(f"[promote] revalidation failed: {report['violations']}")
        return 1
    sandbox_report = sandbox_apply(graph, pid, runtime=rt, promote=True)
```

The `mutate_status=False` parameter is the key design decision here. Between `propose` and `promote`:

1. Other proposals may have been applied, adding new `Behavior` names or altering the protected-type landscape.
2. The proposal's own `validated` stamp could be arbitrarily old.

Re-running the full validator before promote ensures the proposal is still clean against the **current** graph state. Using `mutate_status=False` means this re-check is a pure read—it does not overwrite `status: "validated"` with `status: "validated"` (which would be harmless) and, critically, cannot accidentally overwrite `status: "applied"` if `sandbox_apply` is called concurrently.

The smoke test `test_validate_proposal_mutate_status_false` directly asserts this invariant:

```python
# tests/test_smoke.py:148-159
assert g.get_object(pid).data["status"] == "draft"
report = validate_proposal(g, pid, mutate_status=False)
assert report["ok"]
assert g.get_object(pid).data["status"] == "draft"   # unchanged
```

Sources: [selfgraph/cli.py:83-101](), [tests/test_smoke.py:148-159]()

---

## `sandbox_apply` Enforces `validated` Status

The `sandbox.py` module adds a second guard: it refuses to fork-and-apply any proposal that is not already in `validated` status, regardless of what the caller passes.

```python
# selfgraph/sandbox.py:31-35
if proposal.data.get("status") != "validated":
    raise ValueError(
        f"proposal {proposal_id} has status "
        f"{proposal.data.get('status')!r}; expected 'validated'"
    )
```

This means even if `cmd_promote` skipped the re-check (e.g., called `sandbox_apply` directly), the sandbox would still refuse to apply a `draft` or `rejected` proposal. The smoke test `test_promote_lifecycle_requires_validated_status` validates this by skipping `validate_proposal` entirely and asserting a `ValueError` containing `"validated"` is raised.

Sources: [selfgraph/sandbox.py:31-35](), [tests/test_smoke.py:195-207]()

---

## Complete Validation Flow

```text
cmd_propose / cmd_promote
        │
        ▼
validate_proposal(graph, pid, mutate_status=True|False)
        │
        ├─ fetch PatchProposal object from graph
        ├─ _check_proposal_data()
        │       ├─ _scan_banned(data)          ← whole-payload substring scan
        │       ├─ per-change kind allowlist   ← ALLOWED_KINDS
        │       ├─ protected-type check        ← _PROTECTED_TYPES + approved_by
        │       ├─ permission-escalation check ← add_policy + can_approve
        │       └─ unknown-behavior check      ← bind_behavior vs live graph
        │
        ├─ if mutate_status=True:
        │       graph.patch_object(pid, {status: "validated"|"rejected"})
        │
        └─ return report {ok, checked, violations}
                │
                ▼ (if ok)
        sandbox_apply(promote=False)  ← preview only
                │
                ▼ (cmd_promote, after re-check passes)
        sandbox_apply(promote=True)   ← writes to live graph + status="applied"
```

Sources: [selfgraph/guardrails.py:45-78](), [selfgraph/cli.py:67-101](), [selfgraph/sandbox.py:16-73]()

---

## Summary

`guardrails.py` enforces a closed surface for graph mutation: seven allowed change kinds, a substring banlist against injection payloads, a two-type list protecting the agent's authority substrate, and two extra per-kind rules (no `can_approve` in policies, no phantom behavior bindings). The lifecycle—`draft → validated → applied` or `rejected`—is owned jointly by `guardrails.py` (which stamps `validated`/`rejected`) and `sandbox.py` (which stamps `applied`). The critical safety property is that `cmd_promote` calls `validate_proposal(mutate_status=False)` before `sandbox_apply`, so a stale `validated` marker from a prior run cannot bypass a fresh check against the current graph state. This is tested explicitly in `test_validate_proposal_mutate_status_false` and `test_promote_lifecycle_requires_validated_status`.

Sources: [selfgraph/guardrails.py:45-78](), [selfgraph/cli.py:83-101](), [tests/test_smoke.py:148-207]()

---

## 06. sandbox.py — Fork, Diff, and Promote

> How sandbox_apply forks the SQLite-backed Runtime via Runtime.fork(at_event=...) or falls back to a structural replay on an in-memory graph, applies changes, emits a synthetic smoke TestEvent so newly bound behaviors fire, diffs added_objects and added_relations, and conditionally promotes to the live graph when promote=True. Covers the real-fork vs. in-memory fallback distinction and the single comment in sandbox.py marking where a public projector entry point would live.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/06-sandbox.py-fork-diff-and-promote.md
- Generated: 2026-05-22T06:02:02.189Z

### Source Files

- `selfgraph/sandbox.py`
- `selfgraph/cli.py`
- `tests/test_smoke.py`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [selfgraph/sandbox.py](selfgraph/sandbox.py)
- [selfgraph/cli.py](selfgraph/cli.py)
- [tests/test_smoke.py](tests/test_smoke.py)
- [selfgraph/guardrails.py](selfgraph/guardrails.py)
- [selfgraph/propose.py](selfgraph/propose.py)
</details>

# sandbox.py — Fork, Diff, and Promote

`sandbox.py` is the safe-execution layer that sits between a validated `PatchProposal` and the live graph. Its single public function, `sandbox_apply`, forks the graph into an isolated copy, applies the proposal's changes there, emits a synthetic smoke event so any newly bound behaviors get a chance to fire, and produces a structural diff. Only if the caller explicitly passes `promote=True` are the same changes then written to the live graph and the proposal's status stamped `"applied"`. Until that flag is set, the live graph is never touched.

This separation matters because selfgraph's proposals are LLM-generated patches. The fork-then-diff contract ensures that even a malformed or unexpected proposal cannot corrupt live state without an explicit human promotion step. The file is also the single location in the codebase where a private ActiveGraph API (`Graph._replay_event`) is called, kept isolated from the rest of selfgraph by a clear comment marking where a public projector entry point would eventually live.

---

## Entry Point: `sandbox_apply`

```python
# selfgraph/sandbox.py:16-73
def sandbox_apply(
    graph: Graph,
    proposal_id: str,
    *,
    runtime: Optional[Runtime] = None,
    promote: bool = False,
) -> dict:
```

**Preconditions checked before any fork:**
1. The object at `proposal_id` must exist and have `type == "PatchProposal"`.
2. Its `data["status"]` must be `"validated"` — a `"draft"` or `"rejected"` proposal raises `ValueError` immediately.

Both guards fire before `_build_fork` is called, so no fork is constructed for an unvalidated proposal. The smoke test `test_promote_lifecycle_requires_validated_status` covers this path directly.

Sources: [selfgraph/sandbox.py:28-35]()

---

## Fork Construction: Real Fork vs. In-Memory Fallback

`_build_fork` makes exactly one decision: whether to use `Runtime.fork` (the SQLite-backed real fork) or fall back to a structural event replay into a fresh `Graph`.

```python
# selfgraph/sandbox.py:79-99
def _build_fork(graph: Graph, runtime: Optional[Runtime]):
    store = graph.store
    if runtime is not None and isinstance(store, SQLiteEventStore):
        try:
            last_event = graph.events[-1].id if graph.events else None
            if last_event:
                fork_rt = runtime.fork(at_event=last_event, label="selfgraph-sandbox")
                return fork_rt.graph, f"sqlite-fork@{last_event}"
        except Exception as e:
            print(f"[sandbox] real fork failed, falling back: {e}")

    # Fallback: structural copy by replaying events into a new Graph.
    fresh = Graph(ids=IDGen(), run_id=graph.run_id + "-sandbox")
    _replay_into(fresh, graph.events)
    return fresh, "in-memory-replay"
```

The two paths are summarized below:

| Path | Condition | Fork label | Isolation level |
|---|---|---|---|
| **Real fork** | `runtime` is provided AND `graph.store` is `SQLiteEventStore` AND `graph.events` is non-empty | `sqlite-fork@<event_id>` | True SQLite snapshot via `Runtime.fork(at_event=...)` |
| **In-memory replay** | Any other case (in-memory store, no runtime passed, empty event log, or real fork exception) | `in-memory-replay` | Fresh `Graph` with events projected via `_replay_into` |

The CLI's `cmd_propose` and `cmd_promote` both pass `runtime=rt` (the loaded `Runtime` instance), so in normal CLI use the real fork is taken when the graph is SQLite-backed.

Sources: [selfgraph/sandbox.py:79-99](), [selfgraph/cli.py:69-79](), [selfgraph/cli.py:98-101]()

---

## The Private API Comment

The in-memory fallback calls `Graph._replay_event`, a private method with a leading underscore. The code explicitly documents why:

```python
# selfgraph/sandbox.py:102-108
def _replay_into(target: Graph, events) -> None:
    """Project ``events`` into ``target`` without firing listeners or
    persisting. Calls ``Graph._replay_event``, the documented replay
    entry point used by ``Runtime.load`` and ``Runtime.fork``. If
    ActiveGraph ships a public equivalent later, swap it in here."""
    for ev in events:
        target._replay_event(ev)  # noqa: SLF001 — see docstring
```

This is the **only** private-API call in selfgraph. The module docstring reinforces it: *"This is the only private-API call in selfgraph; isolate it here."* The comment marks exactly where a public projector entry point would be wired once ActiveGraph exposes one.

Sources: [selfgraph/sandbox.py:1-7](), [selfgraph/sandbox.py:92-99](), [selfgraph/sandbox.py:102-108]()

---

## Applying Changes in the Fork

`_apply_changes` iterates the proposal's `changes` list against the fork graph (or the live graph when promoting). It handles all allowed v1 change kinds:

| `kind` | What happens |
|---|---|
| `add_object`, `add_state_bucket`, `add_task`, `add_evaluation` | Calls `graph.add_object(type, data, actor=actor)` |
| `add_relation` | Resolves `from_name`/`to_name` through a local `name_index`, calls `graph.add_relation(...)` |
| `add_policy` | Calls `graph.add_object("Policy", ...)` |
| `bind_behavior` | Calls `graph.add_object("BehaviorBinding", {...})` |
| unknown kinds | **Silently skipped** — a comment notes guardrails should have caught them |

The `name_index` is built from all existing objects at the start of the apply pass, then updated as new objects with `name` fields are added, so relations referencing objects *introduced earlier in the same proposal* resolve correctly.

Sources: [selfgraph/sandbox.py:114-156]()

---

## Synthetic Smoke Event

After changes are applied to the fork, `sandbox_apply` emits a `TestEvent` object:

```python
# selfgraph/sandbox.py:43-49
fork_graph.add_object("TestEvent", {
    "goal": proposal.data.get("goal"),
    "kind": "smoke",
}, actor="sandbox")
```

The inline comment is explicit about what this does and does not do:

> *"Simple test event: emit a synthetic Task.update event so any newly bound behaviors get a chance to fire (in-memory only; we don't spin up a fresh Runtime for the fork in v1)."*

The event is added only to the **fork graph**. No new Runtime is created for the fork, so the event fires in-memory and does not persist or trigger external side effects. The object's `kind: "smoke"` field distinguishes it from real operational events.

Sources: [selfgraph/sandbox.py:43-49]()

---

## Structural Diff

`_diff` computes which objects and relations exist in the fork graph but not in the original, by comparing object/relation ID sets:

```python
# selfgraph/sandbox.py:162-174
def _diff(before: Graph, after: Graph) -> dict:
    before_ids = {o.id for o in before.all_objects()}
    before_rel_ids = {r.id for r in before.all_relations()}
    added_objects = [
        {"id": o.id, "type": o.type,
         "label": o.data.get("name") or o.data.get("goal") or ""}
        for o in after.all_objects() if o.id not in before_ids
    ]
    added_relations = [
        {"id": r.id, "type": r.type, "source": r.source, "target": r.target}
        for r in after.all_relations() if r.id not in before_rel_ids
    ]
    return {"added_objects": added_objects, "added_relations": added_relations}
```

The diff is purely **additive** — the v1 change kinds only add objects and relations; there are no removal or mutation diff keys. The report printed to stdout and returned to the caller is:

```
[sandbox] fork diff: +N objects, +M relations
```

Sources: [selfgraph/sandbox.py:162-174](), [selfgraph/sandbox.py:59-60]()

---

## Promotion: Applying to the Live Graph

When `promote=True` and the report has no failures, `sandbox_apply` re-applies the same changes directly to the original `graph` (not the fork), then patches the proposal's status to `"applied"`:

```python
# selfgraph/sandbox.py:62-69
if promote:
    print(f"[sandbox] promoting proposal to main graph (user approved)")
    _apply_changes(graph, proposal.data["changes"], actor="promote")
    graph.patch_object(
        proposal_id, {"status": "applied"},
        actor="promote",
        rationale="Promoted from sandbox after user approval.",
    )
```

Currently there is no guard inside `sandbox_apply` itself that checks `report["ok"]` before promoting — the function's docstring says *"If `promote=True` (and the report has no failures)"*, but the `ok` flag is set to `True` unconditionally in the current v1 implementation (`"ok": True` at line 57). The safety net is the mandatory `validate_proposal` call upstream: both `cmd_propose` and `cmd_promote` in the CLI call it before reaching `sandbox_apply`, and `cmd_promote` re-validates even after the proposal was already stamped `"validated"` in a prior session.

Sources: [selfgraph/sandbox.py:55-72](), [selfgraph/cli.py:83-101]()

---

## Return Value

`sandbox_apply` returns a single `dict` regardless of whether promotion happened:

```python
{
    "proposal_id": "<id>",
    "fork_label":  "sqlite-fork@<event_id>" | "in-memory-replay",
    "applied_changes": <int>,
    "diff": {
        "added_objects":   [ {"id": ..., "type": ..., "label": ...}, ... ],
        "added_relations": [ {"id": ..., "type": ..., "source": ..., "target": ...}, ... ],
    },
    "ok": True,
}
```

The CLI uses `sandbox["diff"]["added_objects"]` and `sandbox["diff"]["added_relations"]` to print a summary line and to expose the `fork_label` at promotion time.

Sources: [selfgraph/sandbox.py:52-58](), [selfgraph/cli.py:74-79](), [selfgraph/cli.py:99-100]()

---

## Full Lifecycle Sequence

```mermaid
sequenceDiagram
    participant CLI as cli.py (cmd_propose / cmd_promote)
    participant GR as guardrails.py
    participant SB as sandbox.py
    participant Fork as Fork Graph
    participant Live as Live Graph

    CLI->>GR: validate_proposal(graph, pid)
    GR-->>CLI: report {ok, violations}

    CLI->>SB: sandbox_apply(graph, pid, runtime=rt, promote=False/True)
    SB->>SB: check proposal status == "validated"
    SB->>SB: _build_fork(graph, runtime)
    alt SQLite-backed runtime
        SB->>Fork: Runtime.fork(at_event=last_event)
        note right of Fork: label: sqlite-fork@<id>
    else In-memory / no runtime
        SB->>Fork: Graph._replay_event × N
        note right of Fork: label: in-memory-replay
    end
    SB->>Fork: _apply_changes(fork_graph, changes)
    SB->>Fork: add_object("TestEvent", {kind:"smoke"})
    SB->>SB: _diff(graph, fork_graph)
    alt promote=True
        SB->>Live: _apply_changes(graph, changes, actor="promote")
        SB->>Live: patch_object(pid, {status:"applied"})
    end
    SB-->>CLI: report {fork_label, applied_changes, diff, ok}
```

---

## CLI Integration

The two CLI commands that call `sandbox_apply` reflect the two distinct promotion modes:

**`python -m selfgraph propose <goal>`** (`cmd_propose`):
- Validates, then calls `sandbox_apply(..., promote=False)`.
- Prints the diff summary and tells the user what `promote` command to run.
- The live graph is never touched.

**`python -m selfgraph promote <proposal_id>`** (`cmd_promote`):
- Re-validates with `mutate_status=False` (does not overwrite `"validated"` status).
- Calls `sandbox_apply(..., promote=True)`.
- Both the fork diff and the live graph mutation happen in the same call.

Sources: [selfgraph/cli.py:67-101]()

---

## Test Coverage

`tests/test_smoke.py` covers the main behavioral contracts:

| Test | What it verifies |
|---|---|
| `test_proposal_accepted` | In-memory path produces a non-empty `diff["added_objects"]` |
| `test_sandbox_promote_changes_main_graph` | `promote=True` adds objects to the live graph and stamps status `"applied"` |
| `test_sandbox_sqlite_fork_isolates_main_graph` | Real fork label starts with `sqlite-fork@`; live graph object/relation counts are unchanged after `promote=False` |
| `test_promote_lifecycle_requires_validated_status` | `sandbox_apply` raises `ValueError` with `"validated"` in the message for a draft proposal |

The SQLite isolation test is the most important: it confirms that `promote=False` leaves `graph.all_objects()` and `graph.all_relations()` counts unchanged, and that the proposal remains `"validated"` rather than `"applied"`.

Sources: [tests/test_smoke.py:133-207]()

---

## Summary

`sandbox.py` provides the controlled detonation chamber for LLM-generated graph patches. Its `sandbox_apply` function forks the graph (preferring `Runtime.fork` on a SQLite-backed runtime, falling back to `_replay_into` with a private but documented API), applies changes to the isolated copy, emits a smoke `TestEvent` so newly bound behaviors fire in-memory, and diffs the result. The live graph is mutated only when `promote=True` is explicitly passed. The file deliberately concentrates the single private-API call (`Graph._replay_event`) in `_replay_into` with a comment marking where a public projector entry point would slot in once ActiveGraph exposes one — a design decision documented at `selfgraph/sandbox.py:102-108`.

---

## 07. Test Suite & Reproducibility Harness

> What tests/test_smoke.py covers (accept path, banned-token injection, unknown-behavior binding, protected-type addition, disallowed change kind, promote lifecycle) and how the harness/ scripts (run_corpus.py, run_adversarial.py, run_future_event.py, extractor_recall.py, rollback_precondition.py, compare.py, report.py, invariants.py) regenerate the paper result files in harness/results/. Explains the LLM-free invariant enforced by the harness (ANTHROPIC_API_KEY must be unset) and the SELFGRAPH_OBJECTTYPE_MATCH=literal vs. relaxed condition that produces corpus.literal.jsonl vs. corpus.relaxed.jsonl.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/07-test-suite-reproducibility-harness.md
- Generated: 2026-05-22T06:03:03.251Z

### Source Files

- `tests/test_smoke.py`
- `tests/test_harness.py`
- `harness/reproduce.sh`
- `harness/run_corpus.py`
- `harness/invariants.py`
- `REPRODUCE.md`
- `harness/results/CANONICAL_SHAS.txt`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [tests/test_smoke.py](tests/test_smoke.py)
- [tests/test_harness.py](tests/test_harness.py)
- [harness/invariants.py](harness/invariants.py)
- [harness/run_corpus.py](harness/run_corpus.py)
- [harness/run_adversarial.py](harness/run_adversarial.py)
- [harness/rollback_precondition.py](harness/rollback_precondition.py)
- [harness/extractor_recall.py](harness/extractor_recall.py)
- [harness/compare.py](harness/compare.py)
- [harness/report.py](harness/report.py)
- [harness/reproduce.sh](harness/reproduce.sh)
- [REPRODUCE.md](REPRODUCE.md)
- [harness/results/CANONICAL_SHAS.txt](harness/results/CANONICAL_SHAS.txt)
</details>

# Test Suite & Reproducibility Harness

This page covers the two-layer verification system in `activegraph-selfgraph`: a fast pytest suite (`tests/`) that validates the core propose→validate→sandbox lifecycle without any external dependencies, and a measurement harness (`harness/`) that regenerates the result files cited in the paper from a cold start in under one minute. Together they establish that selfgraph's self-modification pipeline is both correct and reproducible on any machine with Python 3.11 and no API key.

The central design constraint is **LLM-freedom**: every measurable number the paper cites comes from a deterministic, network-free pipeline. The harness enforces this as a hard invariant — it refuses to start if `ANTHROPIC_API_KEY` is set — so the canonical SHA-256 fingerprints recorded in `harness/results/CANONICAL_SHAS.txt` can be independently verified without a model subscription.

---

## tests/test_smoke.py — The Unit Test Layer

`tests/test_smoke.py` is the fast, self-contained correctness test for the core `selfgraph` modules. Every test builds a fresh in-memory `Graph` (via `_fresh()`), ingests one or two files with `ingest_paths`, and runs `extract_capabilities(use_llm=False)` to produce a deterministic capability graph before exercising a specific behavior.

Sources: [tests/test_smoke.py:26-46]()

### Covered test scenarios

| Test function | What it checks |
|---|---|
| `test_ingest_and_extract` | `ingest_paths` produces `File` + `Chunk` objects; `extract_capabilities` emits ≥6 `Capability` and ≥4 `AuthorityRule` nodes |
| `test_proposal_accepted` | A benign goal clears `validate_proposal` and `sandbox_apply` adds objects |
| `test_proposal_rejected_when_banned_token_injected` | Manually injecting `subprocess.Popen(['rm', '-rf', '/'])` into a proposal's change list triggers a `banned-token` violation |
| `test_proposal_rejected_for_unknown_behavior` | A `bind_behavior` change referencing a behavior that doesn't exist in the graph triggers `unknown-behavior` |
| `test_proposal_rejected_for_protected_type_add` | Adding `AuthorityRule` or `Capability` objects directly causes ≥2 `protected-type` violations |
| `test_proposal_rejected_for_unknown_change_kind` | `spawn_subprocess` + `add_policy` changes raise both `disallowed-kind` and `permission-escalation` violations |
| `test_sandbox_promote_changes_main_graph` | `sandbox_apply(promote=True)` materializes objects on the live graph and sets the proposal status to `"applied"` |
| `test_validate_proposal_mutate_status_false` | `validate_proposal(mutate_status=False)` returns a passing report without advancing a `"draft"` proposal's status |
| `test_sandbox_sqlite_fork_isolates_main_graph` | With a SQLite-backed `Runtime`, `sandbox_apply(promote=False)` uses the real `Runtime.fork` path (label starts with `sqlite-fork@`) and leaves the live graph byte-identical to before |
| `test_promote_lifecycle_requires_validated_status` | `sandbox_apply(promote=True)` on a still-`"draft"` proposal raises `ValueError` — the lifecycle gate cannot be bypassed |

Sources: [tests/test_smoke.py:53-207]()

### Proposal lifecycle enforced by the tests

```text
         propose_patch_for
               │
          status = "draft"
               │
      validate_proposal()
          ┌────┴────┐
       ok=True   ok=False
          │          │
   status="validated"  status="rejected"
          │
   sandbox_apply(promote=True)
          │
   status="applied"
```

The state transitions are enforced by convention (not a state machine), but `test_promote_lifecycle_requires_validated_status` and `test_validate_proposal_mutate_status_false` together pin both the forward and non-mutating paths. Sources: [tests/test_smoke.py:148-207](), [REPRODUCE.md:259-268]()

---

## tests/test_harness.py — Harness Building Blocks

`tests/test_harness.py` exercises the harness infrastructure itself without running the full corpus (which is slow). Its tests verify that the helper functions used by `run_corpus.py` are deterministic and correct.

| Test | Verified property |
|---|---|
| `test_path_class_runtime_vs_selfgraph` | Paths inside the installed `activegraph` package directory (or `module://activegraph` pseudo-paths) map to `"runtime"`; everything else maps to `"selfgraph"` |
| `test_generate_goal_set_is_deterministic_and_sorted` | Two graphs built identically produce the same goal sequence, sorted by `(node_type, name, template_index)` |
| `test_generate_goal_set_records_provenance` | Every goal row carries `derived_from_node_id`, `derived_from_node_type`, and `derived_from_path_class` |
| `test_classify_change_matches_citation_taxonomy` | `classify_change` returns the exact four category strings: `grounded-in-extracted`, `built-in-scaffold`, `self-authored`, `domain-new` |
| `test_objecttype_match_flag_literal_excludes_runtime_object_types` | Under `SELFGRAPH_OBJECTTYPE_MATCH=literal` the extractor emits zero `ObjectType` nodes whose `source_file_path` lives in the activegraph package |
| `test_objecttype_match_flag_invalid_value_raises` | An unrecognized flag value (e.g. `"lenient"`) raises `ValueError` immediately — a typo cannot silently shift results |
| `test_relaxed_extractor_catches_runtime_object_types` | Under `relaxed` mode, at least one `ObjectType` node (including `"company"` from the diligence pack) is emitted with an activegraph package path |
| `test_run_goal_emits_expected_row_shape` | A full propose→validate→sandbox call via `run_goal` returns a row with all required keys and `live_graph_unchanged=True` |

Sources: [tests/test_harness.py:33-218]()

---

## harness/invariants.py — The LLM-Free Gate

Every harness entry point imports and calls `require_no_llm_env()` as its first action. The function checks whether `ANTHROPIC_API_KEY` is present in the environment and exits with code 64 if it is, unless `SELFGRAPH_HARNESS_ALLOW_LLM=1` is also set (which permits an LLM-augmented variant while printing a loud warning).

```python
# harness/invariants.py:19-52
def require_no_llm_env() -> None:
    if not os.environ.get("ANTHROPIC_API_KEY"):
        return
    if os.environ.get(_OVERRIDE_VAR) == "1":
        # warn and proceed — results will NOT match canonical shas
        return
    # ... structured error message + sys.exit(64)
```

The rationale: `selfgraph/extract.py` has an optional LLM augmentation pass gated on `ANTHROPIC_API_KEY`. If the key is set, the extractor produces an LLM-shaped graph, making the output non-deterministic and breaking both SHA reproducibility and the "no API key required" claim in `REPRODUCE.md`. The `*.meta.json` companion file each run writes carries `"llm_augment_active": false` as an audit stamp.

Sources: [harness/invariants.py:1-52](), [REPRODUCE.md:40-77]()

---

## SELFGRAPH_OBJECTTYPE_MATCH — The A/B Control Variable

The extractor in `selfgraph/extract.py` recognizes `ObjectType` declarations in two modes, selected by the environment variable `SELFGRAPH_OBJECTTYPE_MATCH`:

| Value | Regex active | What it captures | Corpus output |
|---|---|---|---|
| `literal` | `add_object("Cap", ...)` capitalized literal only | Selfgraph-repo ObjectTypes only | `corpus.literal.jsonl` (BEFORE) |
| `relaxed` (default) | Above + `ObjectType(name="...", ...)` constructor calls | Also activegraph runtime pack ObjectTypes | `corpus.relaxed.jsonl` (AFTER) |

Any other value raises `ValueError` immediately — a typo cannot silently produce a shifted result. Sources: [REPRODUCE.md:124-160](), [tests/test_harness.py:111-162]()

The effect on corpus size is significant: the BEFORE condition produces **45 goals** (0 runtime-derived), while the AFTER condition produces **72 goals** (27 runtime-derived + 45 selfgraph-derived). The A/B cleanliness invariant — verified by `harness/compare.py` at the end of every cold run — requires that the selfgraph-derived grounding row be byte-identical across both conditions (27/45 in both). This single-variable guarantee is what makes the runtime-derived 18/27 (66.7%) grounding finding a causal result rather than a confound. Sources: [REPRODUCE.md:141-162]()

---

## harness/run_corpus.py — The Main Measurement Pipeline

`run_corpus.py` is the backbone of the reproducibility harness. It runs the full propose→validate→sandbox measurement loop and streams results to a JSONL file.

### Pipeline steps

```text
build_graph()
  ├── ingest_paths(["selfgraph", "README.md", "demo.py"])
  ├── ingest_module_docs("activegraph", max_submodules=40)
  ├── ingest_paths([activegraph/packs], max_bytes=400_000)
  └── extract_capabilities(use_llm=False)
         │
generate_goal_set(graph)
  └── every Capability + ObjectType × ("monitor {name}", "track {name}", "configure {name}")
      sorted by (node_type, node_name, template_index)
         │
for each goal → run_goal(graph, runtime, goal_row)
  ├── propose_patch_for(graph, goal)
  ├── classify_change() for each change  [origin taxonomy]
  ├── validate_proposal(graph, pid)      [guardrail report]
  └── sandbox_apply(graph, pid, runtime, promote=False)
      ├── assert fork_label starts with "sqlite-fork@"
      └── assert live_graph_unchanged
         │
write corpus.jsonl + run.meta.json
assert fork_violations == [] and isolation_violations == []
```

Sources: [harness/run_corpus.py:1-359]()

### Per-row JSONL schema

Each row records:

- **Goal provenance**: `goal`, `derived_from_node_id`, `derived_from_node_type`, `derived_from_node_name`, `derived_from_source`, `derived_from_path_class`
- **Proposal details**: `proposal_id`, `used_fallback_scaffold`, `n_changes`
- **Origin taxonomy**: `origin_counts` (`grounded-in-extracted`, `built-in-scaffold`, `self-authored`, `domain-new`) and `per_change` list
- **Grounding edges**: `patch_modifies` (list of `{target_id, target_name, target_source, target_path_class}`) and `n_patch_modifies`
- **Guardrail**: `guardrail.ok`, `guardrail.n_violations`, `guardrail.violation_kinds`
- **Sandbox**: `sandbox.fork_label`, `sandbox.fork_path`, `sandbox.n_added_objects`, `sandbox.n_added_relations`, `sandbox.live_graph_unchanged`

Sources: [harness/run_corpus.py:145-255]()

### Determinism measures

Two sources of cross-machine non-determinism were eliminated (documented in `REPRODUCE.md`):
1. Wall-clock fields (`t_start`/`t_end`) were removed from rows.
2. `os.walk` in `ingest_paths` and `pkgutil.walk_packages` in `ingest_module_docs` now sort output before processing, pinning `Object#N` IDs across machines. A `set()` of regex captures in `extract.py` is also sorted.

Sources: [REPRODUCE.md:108-121]()

---

## harness/run_adversarial.py — Guardrail Safety Slice

`run_adversarial.py` mechanically generates unsafe proposals across every violation class and verifies that `validate_proposal` catches them. It does not hand-write adversarial payloads — it enumerates them from the validator's own constants (`_BANNED_TOKENS`, `_PROTECTED_TYPES`, `ALLOWED_KINDS`).

### Generators

| Generator | Method | Violation class |
|---|---|---|
| `gen_banned_token_attempts` | One attempt per token in `guardrails._BANNED_TOKENS`; injects the token into an `add_object` data blob | `banned-token` |
| `gen_unknown_behavior_attempts` | 5 attempts binding synthetic behavior names not present in any ingested source | `unknown-behavior` |
| `gen_protected_type_attempts` | One attempt per protected type (`AuthorityRule`, `Capability`) | `protected-type` |
| `gen_disallowed_kind_attempts` | One attempt using `spawn_subprocess` change kind | `disallowed-kind` |
| `gen_permission_escalation_attempts` | One attempt adding a policy with `can_approve` | `permission-escalation` |

After running all attempts, the script cross-checks `corpus.relaxed.jsonl` for false positives (benign proposals that were incorrectly rejected). The output is a confusion-style table: `n_attempts`, `n_caught`, and `gap` per class. Any under-catch is a **result** recorded in the paper, not a bug silently patched. Sources: [harness/run_adversarial.py:1-315]()

---

## harness/rollback_precondition.py — Promote + Replay Verification

This script measures the precondition for the paper's rollback claim: that every self-modification is a real logged event on the same event log that replay reconstructs.

It reproduces the full `corpus.relaxed` pipeline (same ingest, extract, and `generate_goal_set`), then promotes every guardrail-validated proposal inside an isolated `Runtime.fork`. On the reference machine this covers **n=72 promotions** including all 9 `bind_behavior` proposals. Each fork shares the SQLite file with the main pipeline graph but operates under a distinct `run_id`, so it neither contaminates other trials nor mutates the main pipeline.

Per trial it records:
- `n_promote_events`: number of `actor="promote"` events appended (expected: `n_changes + 1`)
- `all_changes_logged`: whether promote produced enough logged events for every allowed-kind change
- `replay_byte_identical`: whether opening a fresh `SQLiteEventStore` for the fork's `run_id` and replaying events up to (but not including) the first promote-actor event reconstructs a snapshot byte-identical to the pre-promote snapshot

Reference machine result: 72/72 `all_changes_logged`, 72/72 `replay_byte_identical`. Sources: [harness/rollback_precondition.py:1-71](), [REPRODUCE.md:164-205]()

---

## harness/extractor_recall.py — Discovery Recall Measurement

`extractor_recall.py` quantifies the extraction fidelity bottleneck described in §7 of the paper. It uses Python's `ast` module (independent of the extractor's regex) as the ground-truth denominator.

**Behavior denominator**: every function in the activegraph package decorated with `@behavior`, `@llm_behavior`, or `@relation_behavior` (top-level name or attribute tail, with or without a call).

**ObjectType denominator**: union of `add_object("<X>", ...)` first-positional string literals and `ObjectType(name="<X>", ...)` keyword string literals.

The extractor is run twice — once under `SELFGRAPH_OBJECTTYPE_MATCH=literal`, once under `=relaxed` — and recall, missed names, and any runtime-derived false positives (e.g. `hello` from a code-template literal in `activegraph/packs/scaffold.py`) are reported. Output goes to `harness/results/extractor_recall.json`. Sources: [harness/extractor_recall.py:1-80]()

---

## harness/compare.py — A/B Cleanliness Enforcement

`compare.py` reads the BEFORE (`corpus.literal.jsonl`) and AFTER (`corpus.relaxed.jsonl`) files and prints a side-by-side table covering: goal set size by path class, grounding rate overall and split, fallback-scaffold rate, origin mix, and sandbox regression metrics.

The critical output is the **A/B cleanliness invariant** check at the end:

```python
# harness/compare.py:163-173
bs = b["by_class_grounded"].get("selfgraph", (0, 0))
as_ = a["by_class_grounded"].get("selfgraph", (0, 0))
if bs == as_:
    print("→ identical: single-variable A/B holds")
    return 0
# else: MISMATCH — exits non-zero
```

If the selfgraph-derived grounding row differs between conditions, `reproduce.sh` exits non-zero, blocking the result. Sources: [harness/compare.py:156-173]()

---

## harness/report.py — Corpus Aggregate Summary

`report.py` reads a corpus JSONL file (defaulting to `corpus.relaxed.jsonl`) and prints a flat summary covering: corpus shape, grounding rate by path class, fallback-scaffold rate, origin mix, guardrail outcomes with violation kind counts, and sandbox isolation metrics. It is invoked standalone after a run to inspect individual conditions:

```bash
PYTHONPATH=. python -m harness.report
PYTHONPATH=. python -m harness.report harness/results/corpus.literal.jsonl
```

Sources: [harness/report.py:1-143]()

---

## harness/reproduce.sh — Cold-Start Entry Point

`reproduce.sh` is the single command that regenerates all six result files from scratch and verifies them against `CANONICAL_SHAS.txt`. It runs in six numbered steps:

```bash
bash harness/reproduce.sh
```

| Step | Script | Env var | Output |
|---|---|---|---|
| 1 | `harness.run_corpus` | `SELFGRAPH_OBJECTTYPE_MATCH=literal` | `corpus.literal.jsonl` |
| 2 | `harness.run_corpus` | `SELFGRAPH_OBJECTTYPE_MATCH=relaxed` | `corpus.relaxed.jsonl` |
| 3 | `harness.run_adversarial` | relaxed | `adversarial.jsonl` |
| 4 | `harness.rollback_precondition` | relaxed | `rollback.jsonl` |
| 5 | `harness.run_future_event` | relaxed | `future_event.jsonl` |
| 6 | `harness.extractor_recall` | (reads mode internally) | `extractor_recall.json` |

After the six steps, the script runs `harness.compare` (enforcing the A/B invariant) and then checks each regenerated file's `sha256sum | head -c 16` against the values sourced from `CANONICAL_SHAS.txt`. If any SHA mismatches or the A/B invariant fails, it exits non-zero. Sources: [harness/reproduce.sh:1-139]()

### Canonical SHA fingerprints

| File | sha256[:16] | Condition |
|---|---|---|
| `corpus.literal.jsonl` | `57a86e94ba5e211d` | `SELFGRAPH_OBJECTTYPE_MATCH=literal` |
| `corpus.relaxed.jsonl` | `3277086cf459e945` | `SELFGRAPH_OBJECTTYPE_MATCH=relaxed` |
| `adversarial.jsonl` | `09b408bd369dc89d` | relaxed |
| `rollback.jsonl` | `ff7353d410ea7379` | relaxed |
| `future_event.jsonl` | `8418183932468a18` | relaxed |
| `extractor_recall.json` | `82a971df7a9ad03c` | both modes inside |

Sources: [harness/results/CANONICAL_SHAS.txt:1-37]()

---

## Quick-start reference

```bash
# Run all smoke tests (fast, no API key needed)
python -m pytest tests/

# Full cold reproduction — all six result files, SHA check, A/B check
# (must have ANTHROPIC_API_KEY unset)
pip install -r requirements.txt
bash harness/reproduce.sh

# Inspect a single condition's aggregate
PYTHONPATH=. python -m harness.report harness/results/corpus.relaxed.jsonl

# Side-by-side A/B table without running reproduce
PYTHONPATH=. python -m harness.compare \
    harness/results/corpus.literal.jsonl \
    harness/results/corpus.relaxed.jsonl

# Deliberately run an LLM-augmented variant (shas will diverge)
SELFGRAPH_HARNESS_ALLOW_LLM=1 ANTHROPIC_API_KEY=sk-... \
    python -m harness.run_corpus
```

The entire measured loop — `propose`, `guardrails`, `sandbox`, `classify_change`, and all harness scripts — makes zero model calls. The only file that touches a model is `selfgraph/extract.py`, and only when `ANTHROPIC_API_KEY` is set; the harness blocks that path at startup to ensure the canonical SHAs are never silently shaped by an LLM. Sources: [REPRODUCE.md:53-77](), [harness/invariants.py:19-52]()

---

## 08. After 30 Minutes: What You Now Know and Where to Go Next

> A closing map of what a reader should understand after this wiki — the full build→ask→propose→validate→sandbox→promote flow, the safety boundaries (no code authoring, no shell, no external I/O), and the key limitations to keep in mind (fallback scaffold, keyword-only retrieval, no multi-step planning, no UI). Suggests concrete next experiments: run demo.py, inspect graph.db with sqlite3, try a goal that triggers the FALLBACK branch, or add a new regex to extract.py and re-run the harness to verify the sha changes.

- Page Markdown: https://grok-wiki.com/public/wiki/yoheinakajima-activegraph-selfgraph-41747ef30393/pages/08-after-30-minutes-what-you-now-know-and-where-to-go-next.md
- Generated: 2026-05-22T06:05:25.977Z

### Source Files

- `README.md`
- `selfgraph/cli.py`
- `harness/reproduce.sh`
- `REPRODUCE.md`

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [README.md](README.md)
- [selfgraph/cli.py](selfgraph/cli.py)
- [selfgraph/propose.py](selfgraph/propose.py)
- [selfgraph/guardrails.py](selfgraph/guardrails.py)
- [selfgraph/sandbox.py](selfgraph/sandbox.py)
- [selfgraph/extract.py](selfgraph/extract.py)
- [harness/reproduce.sh](harness/reproduce.sh)
- [REPRODUCE.md](REPRODUCE.md)
</details>

# After 30 Minutes: What You Now Know and Where to Go Next

This page is a closing map for anyone who has just worked through the selfgraph wiki. It consolidates the full pipeline into one mental model, names every safety boundary the agent enforces by construction, calls out the known limitations you need to hold in your head before extending anything, and gives you a handful of concrete next moves you can run from the terminal right now.

After reading this you should be able to explain the system to a colleague in under five minutes, predict whether a proposed patch will be accepted or rejected by the guardrails, know exactly which parts of the design are a deliberate sketch rather than a hardened guarantee, and have a clear path into hands-on experimentation.

---

## The Complete Pipeline in One View

The six commands map directly to six phases of a single cycle. Every phase is implemented in a distinct module; none of them calls an LLM (unless `ANTHROPIC_API_KEY` is set, in which case only `extract.py` is affected — and only additively).

```text
┌─────────────────────────────────────────────────────────┐
│  Phase        Command / entry point       Module         │
├─────────────────────────────────────────────────────────┤
│  BUILD        python -m selfgraph build   ingest.py      │
│               (walk repo + introspect     extract.py     │
│                activegraph module)                       │
├─────────────────────────────────────────────────────────┤
│  ASK          python -m selfgraph ask     query.py       │
│               (keyword retrieval from                    │
│                graph nodes)                              │
├─────────────────────────────────────────────────────────┤
│  PROPOSE      python -m selfgraph propose propose.py     │
│               (compose PatchProposal                     │
│                from extracted nodes)                     │
├─────────────────────────────────────────────────────────┤
│  VALIDATE     (runs inside propose/       guardrails.py  │
│                promote automatically)                    │
├─────────────────────────────────────────────────────────┤
│  SANDBOX      (runs inside propose        sandbox.py     │
│                automatically; promote                    │
│                runs it again)                            │
├─────────────────────────────────────────────────────────┤
│  PROMOTE      python -m selfgraph promote cli.py         │
│               (re-validates + applies     sandbox.py     │
│                to live graph)                            │
└─────────────────────────────────────────────────────────┘
            state persists to .selfgraph/graph.db
```

### BUILD → EXTRACT

`cmd_build` calls `ingest_paths` (walks every `.py` and `.md` file in the repo) and `ingest_module_docs` (introspects the `activegraph` package as a synthetic `module://…` corpus), then immediately calls `extract_capabilities`. The result is a set of graph nodes — `Capability`, `API`, `Behavior`, `EventType`, `ObjectType`, `Constraint`, `AuthorityRule` — stored via the SQLite event log at `.selfgraph/graph.db`.

Sources: [selfgraph/cli.py:48-57]()

Extraction is deterministic by default. Two regex paths exist; which one runs is controlled by `SELFGRAPH_OBJECTTYPE_MATCH`. Setting it to `literal` runs only the original capitalized-identifier regex; setting it to `relaxed` (the default in normal use) also runs a constructor-call pattern that captures lowercase pack ObjectTypes like `"company"` or `"document"`. This is the A/B variable the research harness measures.

Sources: [selfgraph/extract.py:36-60]()

### ASK

`answer_question` performs keyword-overlap retrieval over node data fields. There is no embedding model, no BM25, no semantic understanding — the score is literally the count of matching tokens between your question and each node's stored data. The answer cites node IDs so you can inspect the raw objects.

### PROPOSE

`propose_patch_for` scans the graph for `Behavior`, `EventType`, and `ObjectType` nodes and then assembles a `PatchProposal` object with a `changes` list. The logic is:

1. Add an `ObjectType` state bucket derived from the goal text.
2. Add a `Task` object so work is trackable.
3. Try to bind existing `Behavior` nodes whose `on=` event types overlap goal keywords (keyword match, not semantic). If any match, emit `bind_behavior` changes.
4. **If no behavior matched** — the FALLBACK branch — emit the built-in atom/snapshot/`ROLLS_UP_INTO` scaffold instead. The rationale string is prefixed `[FALLBACK]` and scaffold objects carry `"source": "selfgraph-fallback-scaffold"` and `"used_fallback_scaffold": true` on the proposal.
5. Add a scoped `Policy`.
6. Add `Evaluation` criteria.

The FALLBACK branch is not a degraded path you can avoid — it fires whenever the keyword overlap is zero, which is most novel goals. Recognizing `[FALLBACK]` in the output is important: it tells you the structure was templated, not discovered.

Sources: [selfgraph/propose.py:84-121]()

### VALIDATE

`validate_proposal` in `guardrails.py` runs two checks:

| Check | Mechanism | Limitation |
|---|---|---|
| Banned-token scan | Substring search over the entire proposal payload | Easy to evade; also false-positives on docs that *mention* `subprocess` |
| Structural check | Each change must have a `kind` in `ALLOWED_KINDS`; `bind_behavior` names must exist in the graph's `Behavior` nodes; `add_object` for `AuthorityRule`/`Capability` requires `approved_by`; policies may not declare `can_approve` | Only fires on the kinds it knows about |

`ALLOWED_KINDS` is exactly: `add_object`, `add_relation`, `add_policy`, `add_state_bucket`, `add_task`, `add_evaluation`, `bind_behavior`. Any other `kind` value is rejected.

Sources: [selfgraph/guardrails.py:22-126]()

### SANDBOX

`sandbox_apply` forks the runtime before applying any changes. When a SQLite-backed runtime is present it calls `Runtime.fork(at_event=last_event)`, giving a real copy-on-write fork at the current event cursor. When the runtime is in-memory it falls back to replaying events into a fresh `Graph` via the internal `_replay_event` entry point.

After applying the proposal's changes in the fork, it emits a synthetic `TestEvent` object and computes a structural diff (added objects vs. added relations). If `promote=False` (the default during `propose`) the live graph is untouched.

Sources: [selfgraph/sandbox.py:16-73]()

### PROMOTE

`cmd_promote` does **not** trust a cached `validated` status. It calls `validate_proposal(graph, pid, mutate_status=False)` fresh against the current persisted graph, then if that passes calls `sandbox_apply(..., promote=True)` which applies changes to the live graph with `actor="promote"` and stamps the proposal status to `"applied"`.

Sources: [selfgraph/cli.py:83-101]()

---

## PatchProposal Lifecycle

```text
             propose_patch_for()
                     │
                     ▼
                 [ draft ]
                     │
         validate_proposal()
           ┌─────────┴──────────┐
           ▼                    ▼
      [ validated ]         [ rejected ]
           │
    sandbox_apply(promote=False)
    (diff shown; live graph unchanged)
           │
      user runs `promote`
           │
    validate_proposal(mutate_status=False)
    ← re-check against current graph ─────►  [ rejected ]
           │
    sandbox_apply(promote=True)
           │
           ▼
       [ applied ]
```

The transitions are enforced by convention at two call sites, not by a state machine. `sandbox_apply` refuses to act on a proposal that does not carry `status == "validated"`, and `cmd_promote` re-runs validation before calling it.

Sources: [selfgraph/cli.py:83-101](), [selfgraph/sandbox.py:28-35]()

---

## Safety Boundaries: What the Agent Cannot Do

These are hard boundaries enforced by the allowed change-kind list, not soft guidelines:

| Boundary | Mechanism |
|---|---|
| **No code authoring** | There is no change kind that emits a Python function. `bind_behavior` only references a behavior name already discovered in the graph. |
| **No shell execution** | `subprocess`, `os.system`, `popen`, `/bin/sh` are in `_BANNED_TOKENS`. No change kind can invoke a subprocess. |
| **No network or file I/O** | `urllib`, `requests.`, `socket.`, `open(` are banned tokens. Patches write only to the SQLite event store. |
| **No authority escalation** | `AuthorityRule` and `Capability` are `_PROTECTED_TYPES`; adding them without `approved_by` is a guardrail violation. Policies may not declare `can_approve`. |
| **No unknown behaviors** | `bind_behavior` with a name not already in `graph.objects(type="Behavior")` is rejected at validation time. |

Sources: [selfgraph/guardrails.py:21-38](), [README.md:109-130]()

---

## Known Limitations to Keep in Mind

These are documented design choices in v1, not bugs:

**1. FALLBACK scaffold is a built-in template, not discovered structure.**
When `_pick_behavior_bindings` returns an empty list, `propose.py` emits the atom/snapshot/`ROLLS_UP_INTO` shape that ships with selfgraph. Only the trigger `EventType` is observed from the graph; everything else is defaulted. The `[FALLBACK]` prefix in the rationale and `"source": "selfgraph-fallback-scaffold"` on the ObjectType objects are your signals.

**2. Keyword-only retrieval in `ask`.**
`answer_question` scores nodes by token overlap. Synonyms, paraphrasing, and domain terminology that differs from the node's stored text are invisible to it.

**3. No multi-step planning.**
`propose_patch_for` generates one `PatchProposal` per call. It does not chain proposals, reason about dependencies between proposals, or build a campaign. Each goal is treated independently.

**4. Capability extraction is a sketch.**
The regex heuristics miss some `@behavior` decorators, can misclassify API surface, and do not trace dynamic dispatch. The graph is good enough to ground proposals; it is not an authoritative manifest.

**5. Substring banlist is demo-grade.**
`_BANNED_TOKENS` can be evaded (e.g., string concatenation) and false-positives on documentation that mentions banned tokens. A production-grade version would use AST analysis.

**6. No UI, no auth, no remote store.**
The only persistence layer is a local SQLite file at `.selfgraph/graph.db`. There is no web interface, no access control, and no multi-user story.

Sources: [README.md:158-188](), [selfgraph/guardrails.py:27-33]()

---

## Concrete Next Experiments

### 1. Run the scripted demo end-to-end

```bash
pip install -r requirements.txt
python demo.py
```

This runs `build → ask → propose` in sequence against `.selfgraph-demo/graph.db` and prints the grounding trace. Read the output carefully: if the goal triggers the FALLBACK branch you will see `[FALLBACK]` in the rationale and `source: selfgraph-fallback-scaffold` on the added ObjectType nodes.

### 2. Inspect the graph directly after build

```bash
python -m selfgraph build .
sqlite3 .selfgraph/graph.db "SELECT id, type, json(data) FROM objects LIMIT 20;"
```

Every object the agent can see when composing a proposal is in this table. Inspecting it makes the "graph-grounded" claim concrete: you can verify which `Behavior` nodes were extracted and why a particular proposal chose (or failed to choose) a `bind_behavior` change.

### 3. Deliberately trigger the FALLBACK branch

```bash
python -m selfgraph propose "synthesize quarterly earnings into a dashboard"
```

A goal with no keyword overlap with any extracted `Behavior` name or `EventType` will produce a `[FALLBACK]` proposal. Compare this to a goal that does match — for example `"track project updates"` — and observe the structural difference in the `changes` list.

### 4. Add a regex to extract.py and verify the sha changes

Open `selfgraph/extract.py` and add a new pattern under the deterministic extraction section. Then run the reproduce harness:

```bash
unset ANTHROPIC_API_KEY
bash harness/reproduce.sh
```

The harness will report `MISMATCH` for the affected result file because the new pattern changes which nodes get extracted, which changes the object IDs proposals cite, which changes the JSONL bytes. This is the intended behavior: the sha-match table in `REPRODUCE.md` is a reproducibility contract, not a test you are supposed to pass after modifying the extractor.

Sources: [harness/reproduce.sh:23-46](), [REPRODUCE.md:1-30]()

### 5. Explore the adversarial guardrail slice

```bash
SELFGRAPH_OBJECTTYPE_MATCH=relaxed python -m harness.run_adversarial
cat harness/results/adversarial.jsonl | python -c "import sys,json; [print(json.loads(l)['caught'], json.loads(l)['goal'][:60]) for l in sys.stdin]"
```

This runs 28 mechanical injection attempts — banned-token payloads, unknown-behavior bindings, protected-type additions, disallowed change kinds — and shows which guardrail rule fired for each. It is the fastest way to build intuition for where the validator is strict and where it relies on the substring banlist.

---

## Summary

After 30 minutes in this repository, the central insight is that selfgraph's safety guarantee is **structural**, not semantic. The agent cannot propose shell execution, network calls, or new Python functions because there are no change kinds for those operations — the allowed surface is a closed list that the guardrails enforce on every proposal before it reaches the sandbox or the live graph. The LLM (when present) is additive and confined to the `build` phase; the proposal, validation, sandbox, and promote loop are deterministic by construction, which is why `harness/reproduce.sh` can assert byte-identical output across machines. The limitations — fallback scaffold, keyword retrieval, no multi-step planning, sketch-grade extraction — are all first-class design choices documented in the code and in the README, not implementation gaps waiting to be closed.

Sources: [README.md:109-130](), [REPRODUCE.md:52-68]()

---