# Migrate workspaces

> tigerfs migrate --describe/--dry-run, history-format migrations, pre-boundary undo EPERM, and release verification against test/integration migrate tests.

- Repository: timescale/tigerfs
- GitHub: https://github.com/timescale/tigerfs
- Human docs: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3
- Complete Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/llms-full.txt

## Source Files

- `internal/tigerfs/cmd/migrate.go`
- `test/integration/migrate_test.go`
- `docs/release-process.md`
- `docs/adr/019-undo-boundary-via-metadata-table.md`
- `internal/tigerfs/fs/undo_boundary_test.go`

---

---
title: "Migrate workspaces"
description: "tigerfs migrate --describe/--dry-run, history-format migrations, pre-boundary undo EPERM, and release verification against test/integration migrate tests."
---

The `tigerfs migrate` command (`internal/tigerfs/cmd/migrate.go`) connects to a PostgreSQL database, runs an ordered list of self-detecting migrations per user schema, and supports inspection (`--describe`), SQL preview (`--dry-run`), or transactional execution. Migrations upgrade synth workspaces from older TigerFS layouts (user-schema `_app` tables, path-encoded filenames, missing triggers) to the current model: backing tables in `tigerfs`, `parent_id` directories, and optional undo-boundary metadata after the `relational-directories` step.

## When to run migrate

Run migrate when a database was created with an older TigerFS release and still has pending structural changes. Fresh workspaces built via `/.build/` on a current release typically need no migration.

| Signal | Meaning |
|--------|---------|
| `tigerfs migrate … --describe` lists migrations | Work is pending |
| Output is `No pending migrations.` | Schema matches current TigerFS |
| Mount works but nested paths or undo behave oddly on an old DB | Likely skipped `relational-directories` |
| Undo of old log entries fails with permission errors after upgrade | Expected post `history-format-migration` boundary |

<Warning>
Migrate runs DDL in transactions per migration. Take a database backup before applying on production data. Use `--dry-run` first on shared or production databases.
</Warning>

## Command reference

```bash
tigerfs migrate [CONNECTION] [--describe] [--dry-run] [--schema SCHEMA] [--insecure-no-ssl]
```

<ParamField body="CONNECTION" type="string">
Optional connection. Prefix selects backend: `tiger:ID`, `ghost:ID`, or `postgres://…`. If omitted, resolution uses config, env, and `PG*` variables (same as other TigerFS commands).
</ParamField>

<ParamField body="--describe" type="boolean">
List pending migration names, summaries, and affected workspace/table identifiers. Does not execute SQL.
</ParamField>

<ParamField body="--dry-run" type="boolean">
Print planned SQL (semicolon-terminated statements) without executing. Database state is unchanged.
</ParamField>

<ParamField body="--schema" type="string">
User schema to scan (default: database `current_schema()` / `search_path`).
</ParamField>

<ParamField body="--insecure-no-ssl" type="boolean">
Allow non-TLS remote connections (same semantics as other TigerFS commands).
</ParamField>

Default mode (no flags) executes each pending migration inside a single transaction, prints `Running migration: <name>`, then `Migrated N views` per migration. Command timeout is 60 seconds.

<Steps>
<Step title="Inspect pending work">

```bash
tigerfs migrate postgres://user@host/dbname --describe
```

Example output shape:

```text
move-backing-tables: Move backing tables from user schema to tigerfs schema
  - _myapp
relational-directories: Upgrade directory structure for improved performance and undo support
  - myapp
```

</Step>

<Step title="Preview SQL">

```bash
tigerfs migrate postgres://user@host/dbname --dry-run
```

Expect `CREATE SCHEMA`, `ALTER TABLE … SET SCHEMA`, `ALTER TABLE … RENAME`, view recreation, and—for `relational-directories`—`parent_id`, history/log column renames, and metadata `INSERT` for the undo boundary.

</Step>

<Step title="Apply">

```bash
tigerfs migrate postgres://user@host/dbname
```

Re-run `--describe` until you see `No pending migrations.`

</Step>
</Steps>

## Migration pipeline

Migrations run in a fixed order; each has `Detect` (catalog queries) and `Plan` (SQL generation). Later migrations assume earlier ones completed where relevant.

```text
  ┌─────────────────────┐     ┌──────────────────────────┐     ┌────────────────────────────┐
  │ move-backing-tables │ ──► │ relational-directories   │ ──► │ parent-dir-mtime-trigger   │
  │ user._app → tigerfs │     │ parent_id, history/log   │     │ parent directory mtime     │
  │ .app view           │     │ metadata boundary row    │     │ AFTER trigger on children  │
  └─────────────────────┘     └──────────────────────────┘     └────────────────────────────┘
```

| Name | Summary | Detect criteria (abbrev.) |
|------|---------|---------------------------|
| `move-backing-tables` | Move `_name` backing tables to `tigerfs.name`, recreate user-schema view | `_name` table + `tigerfs:` view comment with synth format; not already in `tigerfs` |
| `relational-directories` | Path-encoded filenames → `parent_id` model; history/log column renames; metadata table + boundary | `tigerfs` backing table exists, no `parent_id` column |
| `parent-dir-mtime-trigger` | POSIX-style directory `modified_at` on child changes | Has `parent_id`, missing `trg_<app>_parent_mtime` |

All three are idempotent: detection skips already-migrated objects; re-running with nothing pending prints `No pending migrations.`

### move-backing-tables

Targets legacy layout: backing table `_<app>` in the user schema and view `<app>` with a `tigerfs:` comment. For each match, migration:

1. Ensures `tigerfs` schema exists
2. Moves and renames `_<app>` → `tigerfs.<app>`
3. Drops and recreates the user-schema view pointing at `tigerfs`
4. If `<app>_history` exists in the user schema, moves and renames it to `tigerfs.<app>_history`

Skips views whose comment resolves to native format without history (`FormatNative && !History`).

### relational-directories

Upgrades apps already in `tigerfs` (post move-backing-tables) from ADR-011 path-encoded filenames to ADR-017 `parent_id` directories. Per app, in one transaction:

- Adds `parent_id`, populates it from old path hierarchy, strips filenames to leaf names
- Switches `id` default to `uuidv7()`, adds FK and `UNIQUE NULLS NOT DISTINCT (parent_id, filename, filetype)`
- Recreates the user-schema view (PostgreSQL `SELECT *` views do not pick up new columns automatically)
- If history exists: renames `id`→`file_id`, `_history_id`→`version_id`, `_operation`→`operation`; maps `UPDATE`/`DELETE` to `edit`/`delete`; rebuilds archive trigger
- If log exists: renames `history_id`→`version_id`; maps log types `insert`→`create`, `update`→`edit`; updates CHECK constraint
- Creates `tigerfs.<app>_metadata` if missing and inserts exactly one `history-format-migration` row (`WHERE NOT EXISTS` guard)

<Info>
The boundary `INSERT` runs at the end of this migration so its `entry_id` (UUIDv7) sorts after every pre-migration `log_id`. That lexical ordering is what the undo engine uses to block pre-boundary targets.
</Info>

### parent-dir-mtime-trigger

Adds `GenerateParentDirMtimeTriggerSQL` for apps on the parent-pointer model so directory `modified_at` updates when children are inserted, deleted, or moved—important for NFS directory listing freshness.

## History-format migration and undo boundary

ADR-019 introduces `tigerfs.<app>_metadata` as a sibling of `_history`, `_log`, and `_savepoint`. The `relational-directories` migration inserts one row:

| Field | Value |
|-------|--------|
| `subject` | `history-format-migration` (`synth.SubjectHistoryFormatMigration`) |
| `description` | User-facing hint when undo is refused |
| `payload` | `{"from":"0.6","to":"0.7","reason":"parent-pointer model"}` |

**Why undo is blocked:** migration sets history `parent_id` from the *current* source row, not the historical directory at edit time. Pre-migration log entries remain structurally valid but undo could restore content while leaving the file in the wrong directory.

```mermaid
sequenceDiagram
    participant User
    participant Mount as fs.Operations / mount
    participant Undo as undo.go checkBoundary
    participant Meta as tigerfs.app_metadata

    User->>Mount: .undo/.../.apply (target log_id)
    Mount->>Undo: checkBoundary(schema, app, targetLogID)
    Undo->>Meta: cached metadata rows (mount lifetime)
    alt targetLogID < blocker.entry_id
        Undo-->>Mount: FSError ErrPermission + Hint=description
        Mount-->>User: EPERM (FUSE/NFS) + stderr hint
    else targetLogID >= blocker.entry_id
        Undo-->>Mount: nil (proceed to undo SQL)
    end
```

### Pre-boundary undo: EPERM and hints

`checkBoundary` in `internal/tigerfs/fs/undo.go` runs before `ExecuteUndoSingle`, `ExecuteUndoToLogID`, and `ExecuteUndoToSavepoint` database work. Rules:

| Condition | Result |
|-----------|--------|
| No metadata rows (fresh 0.7+ install) | Allow undo (fast path) |
| `targetLogID < entry_id` for a blocking subject | Refuse: `ErrPermission`, `Hint` = row `description` |
| `targetLogID == entry_id` | Allow (boundary row itself is not a blocked target) |
| `targetLogID > entry_id` | Allow |

Blocking subjects are hard-coded in `blockingSubjects` (currently only `history-format-migration`). FUSE maps `ErrPermission` to `syscall.EPERM`; NFS maps to `os.ErrPermission`.

<RequestExample>

```bash
# After upgrading a v0.6 workspace — undo of an old log id
touch /mnt/db/myapp/.undo/id/<pre-migration-log-uuid>/.apply
```

</RequestExample>

<ResponseExample>

```text
# stderr (structured logging) includes the metadata description as hint
# shell
apply: Operation not permitted   # EPERM
```

</ResponseExample>

Pre-migration history remains readable via `.log/` and `.history/`; only `.undo/` apply paths refuse. Post-migration edits, savepoints, and undo-to-log-id targets after the marker behave normally (covered in `verifyMigrationUndoBoundary` in integration tests).

<Check>
Fresh installs: `/.build/` creates an empty `_metadata` table. With zero rows, `checkBoundary` returns immediately and undo is unrestricted (`TestSynth_FreshInstall_MetadataFastPath`).
</Check>

## Release verification

When a release adds or changes migrations, extend the release checklist (`docs/release-process.md` §1b):

<Steps>
<Step title="Start Postgres 18 + TimescaleDB">

```bash
docker run --rm -d --name migrate-test -e POSTGRES_PASSWORD=test \
  -p 15433:5432 timescale/timescaledb-ha:pg18
sleep 5
```

</Step>

<Step title="Seed pre-release fixture (if applicable)">

Use a database snapshot or SQL fixture representing the previous release layout before running migrate.

</Step>

<Step title="Run migrate describe → dry-run → apply">

```bash
CONN='postgres://postgres:test@127.0.0.1:15433/postgres?sslmode=disable'

tigerfs migrate "$CONN" --describe
tigerfs migrate "$CONN" --dry-run
tigerfs migrate "$CONN"
```

</Step>

<Step title="Run integration migrate tests">

```bash
TEST_DATABASE_URL="$CONN" go test -count=1 -timeout 600s ./test/integration/... -run '^TestSynth_Migrate'
```

Tests in `test/integration/migrate_test.go`:

| Test | Covers |
|------|--------|
| `TestSynth_MigrateDetectAndExecute` | `move-backing-tables`: describe, dry-run, execute, idempotency |
| `TestSynth_MigrateWithHistory` | Backing + `_history` table move |
| `TestSynth_MigrateAddParentPointer` | `relational-directories`, FS ops on migrated tree, undo boundary block/allow |
| `TestSynth_MigrateAddParentDirMtimeTrigger` | Trigger install and mtime behavior |
| `TestSynth_MigrateParentDirMtimeTrigger_NotNeeded` | Fresh build skips trigger migration |
| `TestSynth_FreshInstall_MetadataFastPath` | Empty metadata, undo allowed |

Unit tests for boundary logic: `go test ./internal/tigerfs/fs/... -run TestCheckBoundary` (`undo_boundary_test.go`).

For v0.7-style releases, manually confirm pre-boundary `.undo/…/.apply` returns EPERM with the metadata `description` as the logged hint.

</Step>
</Steps>

<Note>
`docs/release-process.md` suggests `-run TestMigrate`; the repository’s migrate integration tests use the `TestSynth_Migrate` prefix. Use `-run '^TestSynth_Migrate'` (or a narrower name) to match actual test names.
</Note>

## Troubleshooting

| Symptom | Likely cause | Action |
|---------|----------------|--------|
| `No pending migrations.` but mount errors on old DB | Wrong `--schema` | Pass `--schema` for the schema that owns `tigerfs:` views |
| `migration …: failed to execute SQL` | DDL conflict, permissions, or extension missing | Read failed SQL in error; fix DB state; restore from backup if needed |
| Undo EPERM with long hint after upgrade | Pre-boundary log target | Use `.log/<id>/before` or `.history/`; do not `.undo/` pre-migration ids |
| Duplicate boundary rows | Should not happen | `INSERT … WHERE NOT EXISTS` keeps one marker; if count ≠ 1, inspect `tigerfs.<app>_metadata` manually |
| `relational-directories` not listed | Tables still in user schema | Run `move-backing-tables` first (pipeline order) |
| History-enabled app missing TimescaleDB | Hypertable expectations | Install TimescaleDB extension before migrate on history workspaces |

## Related pages

<CardGroup>
<Card title="History, savepoints, and undo" href="/history-savepoints-undo">
`.log`, `.history`, `.undo`, savepoints, and how the migration boundary interacts with undo.
</Card>
<Card title="CLI reference" href="/cli-reference">
Full `migrate` entry alongside mount, build, and connection commands.
</Card>
<Card title="File-first workspaces" href="/file-first-workspaces">
Workspace layout before and after `parent_id` migration.
</Card>
<Card title="Develop and test" href="/develop-and-test">
`go test`, integration testcontainers, and pre-commit checks.
</Card>
<Card title="Error codes" href="/error-codes">
Mapping `fs.ErrPermission` to EPERM and reading stderr hints.
</Card>
</CardGroup>