# Schema migration

> schemaVersion vs binary VERSION, SchemaVersionMismatchError recovery, loopany migrate discovery, and v0.1.0→v0.2.0 script-driven workspace transforms.

- Repository: superdesigndev/loopany
- GitHub: https://github.com/superdesigndev/loopany
- Human docs: https://grok-wiki.com/public/docs/superdesigndev-loopany-97bd9ab97ae8
- Complete Markdown: https://grok-wiki.com/public/docs/superdesigndev-loopany-97bd9ab97ae8/llms-full.txt

## Source Files

- `src/version.ts`
- `src/core/engine.ts`
- `src/commands/migrate.ts`
- `skills/migrations/v0.1.0-to-v0.2.0/SKILL.md`
- `skills/migrations/README.md`
- `test/migration-v0.1-to-v0.2.e2e.test.ts`
- `test/migration-framework.test.ts`

---

---
title: "Schema migration"
description: "schemaVersion vs binary VERSION, SchemaVersionMismatchError recovery, loopany migrate discovery, and v0.1.0→v0.2.0 script-driven workspace transforms."
---

Every `loopany` command except `migrate` and `doctor` calls `bootstrap()` in `src/core/engine.ts`, which compares `$LOOPANY_HOME/config.yaml` `schemaVersion` to the compile-time `SCHEMA_VERSION` constant in `src/version.ts`. A mismatch throws `SchemaVersionMismatchError` and blocks artifact, graph, and search operations until you run the Bun scripts under `skills/migrations/v<from>-to-v<to>/` — the CLI only discovers and documents those scripts; it does not execute them.

## Two independent version axes

| Identifier | Location | Meaning |
| --- | --- | --- |
| `VERSION` | `src/version.ts` | Semver of the **binary** (`loopany --version`). Bumps on any release. |
| `SCHEMA_VERSION` | `src/version.ts` | On-disk **workspace format** the binary expects. Bumped only when kinds, frontmatter, or references change in a way that needs migration. |
| `schemaVersion` | `$LOOPANY_HOME/config.yaml` | The workspace’s **claimed** format version. |

Both `VERSION` and `SCHEMA_VERSION` are currently `0.2.0`, but they are documented to drift independently: you can ship CLI `0.2.1` without changing workspace layout, or bump `SCHEMA_VERSION` to `0.3.0` and ship a migration skill without a binary feature release.

<Note>
`src/core/search-store.ts` defines a separate SQLite `SCHEMA_VERSION` for `search.db`. That index schema is unrelated to workspace `config.yaml#schemaVersion`.
</Note>

### Legacy default

If `config.yaml` exists but omits `schemaVersion`, `Config.load()` assumes `0.1.0` (`ASSUMED_LEGACY_VERSION` in `src/core/config.ts`). Pre-framework workspaces never wrote the field. `loopany init` always writes the current `SCHEMA_VERSION` when it creates a new config file; it does not overwrite an existing `config.yaml`.

## Bootstrap guard and recovery

```mermaid
stateDiagram-v2
  [*] --> Stale: schemaVersion ≠ SCHEMA_VERSION
  Stale --> Blocked: bootstrap() without skip
  Blocked --> MismatchError: SchemaVersionMismatchError
  MismatchError --> Discover: loopany migrate
  Discover --> RunScripts: bun run skills/migrations/.../scripts/*.ts --apply
  RunScripts --> Current: schemaVersion = SCHEMA_VERSION
  Current --> [*]: normal CLI commands succeed
  Stale --> DoctorReport: loopany doctor (skipVersionCheck)
  DoctorReport --> Discover: schema version check fails with migrate hint
```

On mismatch, `SchemaVersionMismatchError` embeds the workspace version, the binary’s expected version, the path to the matching migration skill, and the suggested CLI name:

```
Workspace schema is v0.1.0 but this binary expects v0.2.0.
Read `skills/migrations/v0.1.0-to-v0.2.0/SKILL.md`
and run `loopany migrate v0.1.0-to-v0.2.0`.
```

`src/cli.ts` catches this error class, prints the message to stderr, and exits with code `1` — the same path used for `WorkspaceNotFoundError` and Zod validation failures.

### Commands that bypass the guard

| Surface | Bypass mechanism |
| --- | --- |
| `loopany migrate` | `bootstrap({ skipVersionCheck: true })` |
| `loopany doctor` | `bootstrap({ skipVersionCheck: true })` — reports mismatch as a **fail** check instead of crashing |
| Migration scripts | Set `LOOPANY_SKIP_VERSION_CHECK=1` in the environment (also honored by `bootstrap`) |

<Warning>
Do not leave `LOOPANY_SKIP_VERSION_CHECK=1` in your shell profile. It disables the guard for every command in that session.
</Warning>

After migration completes and `schemaVersion` matches `SCHEMA_VERSION`, regular commands (`artifact list`, `refs`, `search`, etc.) bootstrap normally again.

## `loopany migrate` — discovery only

`src/commands/migrate.ts` scans `skills/migrations/` next to the repo’s `skills/loopany-core/`. Directory names must match `v<semver>-to-v<semver>` (full three-part semver, e.g. `v0.1.0-to-v0.2.0`).

| Invocation | Behavior |
| --- | --- |
| `loopany migrate` | Prints workspace `schemaVersion`, binary `SCHEMA_VERSION`, all known migrations, and **next** migration where `from` equals the workspace version |
| `loopany migrate <name>` | Prints migration metadata, ordered `scripts/*.ts` paths, and the full `SKILL.md` body |

The command **does not** run migration logic. Scripts stay standalone so you can dry-run, apply step-by-step, inspect intermediate disk state, and abort without the binary partially mutating runtime state through coupled `--run` flags.

<RequestExample>

```bash
loopany migrate
```

</RequestExample>

<ResponseExample>

```text
Workspace schema: v0.1.0
Binary expects:   v0.2.0

Next migration:   v0.1.0-to-v0.2.0
                  loopany migrate v0.1.0-to-v0.2.0

Available migrations (1):
  v0.1.0-to-v0.2.0
```

</ResponseExample>

On an up-to-date workspace, list output includes `Up to date — no migration needed.` If the workspace version has no matching `from` directory, you get `No migration found from v…` (file an issue — the binary should ship a chain for every supported jump).

## Migration pack layout

:::files
skills/migrations/
  README.md
  v<from>-to-v<to>/
    SKILL.md
    scripts/
      01-<step>.ts
      ...
      05-bump-version.ts
      06-refresh-kinds.ts
:::

Authoring rules from `skills/migrations/README.md`:

- **One migration = one minor version jump** — chain `v0.1.0-to-v0.2.0` then `v0.2.0-to-v0.3.0` instead of multi-hop scripts.
- **Scripts are standalone** — no imports from `src/core/`; they parse the **old** on-disk format the current binary may no longer understand.
- **Dry-run by default** — pass `--apply` to commit writes.
- **Idempotent** — reruns should be no-ops once the workspace is at the target shape.
- **Workspace-rooted** — read `LOOPANY_HOME` (default `~/loopany`), not the process cwd.
- **Final step must bump `schemaVersion`** — e.g. `05-bump-version.ts` writes `0.2.0` to `config.yaml`.

The skill (`SKILL.md`) carries rationale, pre-flight, ordered steps, rollback, and verification. The CLI is an index; the agent (or operator) runs `bun run` on each script.

## v0.1.0 → v0.2.0 transforms

The shipped migration `skills/migrations/v0.1.0-to-v0.2.0/` is the first major workspace simplification: slug-as-ID storage, flat `artifacts/<dirName>/` paths, camelCase frontmatter, auto timestamps, and a `journal` kind as the time spine.

| Area | v0.1.0 | v0.2.0 |
| --- | --- | --- |
| Kind defs | `idPrefix`, `bodyMode`, `storage`, `idStrategy` | Optional `slugLayout`; slugs are global IDs |
| Artifact paths | `artifacts/{YYYY-MM}/` for date-bucketed kinds | `artifacts/<dirName>/<slug>.md` |
| IDs | Prefixed (`tsk-20260428-091500`) | Bare slug (`20260428-091500`) |
| Frontmatter | snake_case (`check_at`) | camelCase (`checkAt`) + `createdAt` / `updatedAt` |
| Provenance | — | `_backfilled: true` on migrated rows |
| Time index | Implicit in paths | `artifacts/journal/<YYYY>/<YYYY-MM-DD>.md` backfilled from `createdAt` |
| Graph | `references.jsonl` with old IDs | Rewritten via `.migration-id-map.json` |

Crewlet-specific kinds and fields are intentionally **not** migrated — only loopany core shapes.

### Script pipeline

| Step | Script | Role |
| --- | --- | --- |
| 1 | `01-snapshot.ts` | Read-only counts, collision detection (exit `1` if two old IDs map to the same slug) |
| 2 | `02-transform-artifacts.ts` | Rewrite frontmatter, flatten paths, rewrite `[[wiki-links]]`, write `.migration-id-map.json` |
| 3 | `03-rebuild-references.ts` | Rewrite `references.jsonl` using the id map; keeps `references.jsonl.v0.1.bak` |
| 4 | `04-build-journal.ts` | One journal file per distinct `createdAt` date |
| 5 | `05-bump-version.ts` | Sets `config.yaml` `schemaVersion: 0.2.0` — **gate** for bootstrap |
| 6 | `06-refresh-kinds.ts` | Overwrites `kinds/*.md` from bundled v0.2 defs (`init` never overwrites existing kind files) |

Shared helpers live in `scripts/_lib.ts` and intentionally duplicate v0.1 parsing logic.

<Steps>
<Step title="Pre-flight snapshot">

```bash
cd $LOOPANY_HOME
git init -q 2>/dev/null || true
git add -A && git commit -q -m "pre-migration snapshot v0.1.0" || true
loopany doctor > /tmp/loopany-pre-migration-doctor.txt 2>&1 || true
```

Fix any existing `doctor` **fail** checks before migrating — scripts assume a healthy v0.1 workspace.

</Step>
<Step title="Discover migration">

```bash
loopany migrate
loopany migrate v0.1.0-to-v0.2.0   # prints SKILL.md + script list
```

</Step>
<Step title="Run scripts (dry-run, then apply)">

```bash
SKILL=skills/migrations/v0.1.0-to-v0.2.0/scripts
export LOOPANY_HOME=~/loopany
export LOOPANY_SKIP_VERSION_CHECK=1

bun run $SKILL/01-snapshot.ts
bun run $SKILL/02-transform-artifacts.ts
bun run $SKILL/02-transform-artifacts.ts --apply
bun run $SKILL/03-rebuild-references.ts --apply
bun run $SKILL/04-build-journal.ts --apply
bun run $SKILL/05-bump-version.ts --apply
bun run $SKILL/06-refresh-kinds.ts --apply
```

</Step>
<Step title="Verify">

```bash
unset LOOPANY_SKIP_VERSION_CHECK
loopany doctor
loopany artifact list --kind task
loopany artifact list --kind journal
```

Expect `schema version: ok`, no `tsk-`/`sig-` prefixes on IDs, flat `artifacts/` dirs (no `{YYYY-MM}/` buckets), and journal entries under `artifacts/journal/<year>/`.

</Step>
</Steps>

### Slug collisions

If step 02 would assign the same slug to two artifacts (e.g. `mis-self` and `prs-self` both → `self`), `01-snapshot.ts` fails loud. Rename one source file before `--apply`, then rerun from snapshot.

### Rollback

```bash
cd $LOOPANY_HOME
git reset --hard HEAD
rm -f .migration-id-map.json
```

All migration mutations stay inside the workspace tree; the pre-flight git commit is the primary recovery path.

<Info>
`loopany init` is idempotent for existing files — it will not refresh stale `kinds/*.md`. Step `06-refresh-kinds.ts` exists precisely because post-migration kind defs must match v0.2 validators.
</Info>

## Authoring a future migration

When changing on-disk format:

1. Bump `SCHEMA_VERSION` in `src/version.ts` and add `skills/migrations/v<from>-to-v<to>/` with `SKILL.md` + numbered scripts.
2. Use **minor** bumps for additive or transformable changes; reserve **major** for breaks no script can bridge.
3. End with a script that calls `config.setSchemaVersion('<to>')` or equivalent YAML write.
4. Add `test/migration-*.e2e.test.ts` that builds a fixture old workspace, runs scripts via `bun run` with `LOOPANY_HOME` set, and asserts `loopany doctor` passes.

There are **no backwards-compat shims** in the binary for old formats — migration is the only supported forward path once a new `SCHEMA_VERSION` ships.

## Test coverage

| Test file | What it proves |
| --- | --- |
| `test/migration-framework.test.ts` | `schemaVersion` load/default/persist, init writes version, bootstrap guard, `LOOPANY_SKIP_VERSION_CHECK`, `migrate` list/describe, doctor surfaces mismatch |
| `test/migration-v0.1-to-v0.2.e2e.test.ts` | Full six-script pipeline on a synthetic v0.1 workspace; id map, paths, camelCase, refs rewrite, journal backfill, kind refresh, doctor clean, CLI unblocked |

## Related pages

<CardGroup>
<Card title="Configuration reference" href="/configuration-reference">
`schemaVersion`, `LOOPANY_HOME`, and `LOOPANY_SKIP_VERSION_CHECK` in config and environment.
</Card>
<Card title="CLI reference" href="/cli-reference">
`migrate` and `doctor` flags, JSON output, and command inventory.
</Card>
<Card title="Artifacts and workspace" href="/artifacts-and-workspace">
v0.2 slug-as-id layout, `artifacts/<dirName>/`, and journal storage after migration.
</Card>
<Card title="Doctor and troubleshooting" href="/doctor-and-troubleshooting">
Schema version check in doctor output and recovery when commands refuse to run.
</Card>
<Card title="Workspace setup" href="/workspace-setup">
Fresh `loopany init` writes current `schemaVersion` without migrating.
</Card>
</CardGroup>