# TigerFS Documentation
> Reference for mounting PostgreSQL as a versioned filesystem: CLI, dot-directory capabilities, file-first workspaces, data-first pipelines, agent skills, and platform adapters (FUSE/NFS).
## Context Links
- [Agent index](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/llms.txt)
- [Human interactive docs](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3)
- [GitHub repository](https://github.com/timescale/tigerfs)
## Repository Metadata
- Repository: timescale/tigerfs
- Generated: 2026-06-03T08:03:54.777Z
- Updated: 2026-06-03T08:04:10.955Z
- Runtime: Grok CLI
- Format: Documentation
- Pages: 25
## Page Index
- 01. [Overview](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/01-overview.md) - What TigerFS exposes (mount + dot-directories), file-first vs data-first entry paths, runtime assumptions (PostgreSQL, FUSE/NFS), and the first routes to read.
- 02. [Installation](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/02-installation.md) - Install paths (curl installer, go install, releases), platform prerequisites (Linux FUSE, macOS NFS, Docker), and verification commands.
- 03. [Quickstart](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/03-quickstart.md) - First successful mount (postgres:// or tiger:), explore with ls/cat, demo.sh workflows, and expected success signals.
- 04. [Filesystem as API](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/04-filesystem-as-api.md) - Mount hierarchy, schema/table/row/column path model, dot-directory control surface, and how fs.Operations maps paths to SQL.
- 05. [File-first and data-first](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/05-file-first-and-data-first.md) - When a mount path is a workspace vs a table, .build/.format vs raw tables, .tables/ backing schema, and mode detection rules.
- 06. [Capability directories](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/06-capability-directories.md) - Pipeline path grammar (.by, .filter, .order, .first, .last, .export), chaining limits, SQL pushdown, and PathType resolution in path.go.
- 07. [Synthesized apps](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/07-synthesized-apps.md) - Workspace creation via .build/, markdown/plaintext formats, backing tables in tigerfs schema, views, and column/frontmatter mapping.
- 08. [Consistency and caching](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/08-consistency-and-caching.md) - Cross-mount read freshness (no content cache), metadata/stat/path cache TTLs, invalidation on writes, and pipeline cache key isolation.
- 09. [Platform backends](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/09-platform-backends.md) - Linux FUSE vs macOS in-process NFS (go-nfs), write/commit model on NFS, mount registry, and adapter delegation to fs.Operations.
- 10. [File-first workspaces](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/10-file-first-workspaces.md) - Create workspaces, read/write markdown and plaintext, directories as state, frontmatter semantics, and backing-table access via .tables/.
- 11. [Data-first exploration](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/11-data-first-exploration.md) - Row/column reads, index navigation (.by), pagination (.first/.last/.sample), PATCH writes, import/export, and large-table safety limits.
- 12. [History, savepoints, and undo](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/12-history-savepoints-and-undo.md) - Enable history on workspaces, .history/.log paths, savepoint JSON, .undo preview/apply, per-user filters, auto-savepoints, and migration boundaries.
- 13. [Agent skills](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/13-agent-skills.md) - Bundled skills/tigerfs pack, install-time staging to agent config dirs, SKILL.md navigation (files, data, ops, recipes), and mount-time copy behavior.
- 14. [Cloud backends](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/14-cloud-backends.md) - tiger: and ghost: prefix resolution, create/fork/list flows, default_backend config, and Tiger CLI credential requirements.
- 15. [DDL staging](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/15-ddl-staging.md) - .create/.modify/.delete staging dirs, sql/.test/.commit control files, index DDL via .indexes/, grace period, and errno on validation failures.
- 16. [CLI reference](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/16-cli-reference.md) - All cobra commands (mount, unmount, stop, status, info, list, create, fork, migrate, test-connection, config, version), flags, and argument patterns.
- 17. [Configuration reference](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/17-configuration-reference.md) - Config struct fields, ~/.config/tigerfs/config.yaml, TIGERFS_* env vars, precedence, TLS enforcement, and mount-specific overrides.
- 18. [Data formats reference](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/18-data-formats-reference.md) - TSV, CSV, JSON, YAML row encoding, PATCH semantics, NULL representation, type extensions (.txt/.json/.bin), and binary_encoding options.
- 19. [Connection reference](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/19-connection-reference.md) - postgres:// URIs, PG* env vars, password_command, .pgpass, sslmode=require rules, localhost exemptions, and pool settings.
- 20. [Error codes](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/20-error-codes.md) - fs.ErrorCode to POSIX errno mapping, structured zap hints on failure, constraint violations, and reading stderr when tools report EINVAL/EACCES.
- 21. [Demo environment](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/21-demo-environment.md) - scripts/demo workflows (Docker vs macOS native), seed data, demo.sh start/shell/stop, and sample exploration commands.
- 22. [Workflow recipes](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/22-workflow-recipes.md) - Copy-paste file-first patterns: kanban (todo/doing/done + mv), knowledge base, session context, from skills/tigerfs/recipes.md.
- 23. [Migrate workspaces](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/23-migrate-workspaces.md) - tigerfs migrate --describe/--dry-run, history-format migrations, pre-boundary undo EPERM, and release verification against test/integration migrate tests.
- 24. [Develop and test](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/24-develop-and-test.md) - Build commands, go test ./... and integration testcontainers, pre-commit checks, stress runner, and release-process checklist.
- 25. [Troubleshooting](https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/25-troubleshooting.md) - Stale mounts, force unmount, NFS monotonicity warnings in stress runs, logging --debug, .refresh metadata clear, and common errno recovery.
## Source File Index
- `.github/workflows/test.yml`
- `.goreleaser.yaml`
- `CHANGELOG.md`
- `CLAUDE.md`
- `cmd/tigerfs/main.go`
- `docs/adr/001-fuse-library.md`
- `docs/adr/003-ddl-staging-pattern.md`
- `docs/adr/013-backend-prefix-scheme.md`
- `docs/adr/014-query-reduction-caching.md`
- `docs/adr/016-undo-and-recovery.md`
- `docs/adr/019-undo-boundary-via-metadata-table.md`
- `docs/data-first.md`
- `docs/file-first.md`
- `docs/history.md`
- `docs/quickstart.md`
- `docs/release-process.md`
- `docs/spec.md`
- `go.mod`
- `internal/tigerfs/backend/ghost.go`
- `internal/tigerfs/backend/resolve.go`
- `internal/tigerfs/backend/tiger.go`
- `internal/tigerfs/cmd/cmd_test.go`
- `internal/tigerfs/cmd/config.go`
- `internal/tigerfs/cmd/create.go`
- `internal/tigerfs/cmd/fork.go`
- `internal/tigerfs/cmd/migrate.go`
- `internal/tigerfs/cmd/mount.go`
- `internal/tigerfs/cmd/root.go`
- `internal/tigerfs/cmd/stop.go`
- `internal/tigerfs/cmd/test_connection.go`
- `internal/tigerfs/cmd/unmount.go`
- `internal/tigerfs/config/config_test.go`
- `internal/tigerfs/config/config.go`
- `internal/tigerfs/db/connection.go`
- `internal/tigerfs/db/password.go`
- `internal/tigerfs/db/pipeline.go`
- `internal/tigerfs/db/query.go`
- `internal/tigerfs/db/tls.go`
- `internal/tigerfs/format/convert.go`
- `internal/tigerfs/format/csv.go`
- `internal/tigerfs/format/json.go`
- `internal/tigerfs/format/yaml.go`
- `internal/tigerfs/fs/auto_savepoint_test.go`
- `internal/tigerfs/fs/build.go`
- `internal/tigerfs/fs/constants.go`
- `internal/tigerfs/fs/context.go`
- `internal/tigerfs/fs/ddl.go`
- `internal/tigerfs/fs/errors_test.go`
- `internal/tigerfs/fs/errors.go`
- `internal/tigerfs/fs/extensions.go`
- `internal/tigerfs/fs/history.go`
- `internal/tigerfs/fs/metadata_cache.go`
- `internal/tigerfs/fs/operations.go`
- `internal/tigerfs/fs/path_cache.go`
- `internal/tigerfs/fs/path.go`
- `internal/tigerfs/fs/staging.go`
- `internal/tigerfs/fs/synth/build.go`
- `internal/tigerfs/fs/synth/format.go`
- `internal/tigerfs/fs/synth/markdown.go`
- `internal/tigerfs/fs/synth/plaintext.go`
- `internal/tigerfs/fs/undo_boundary_test.go`
- `internal/tigerfs/fs/undo.go`
- `internal/tigerfs/fs/write.go`
- `internal/tigerfs/fuse/adapter.go`
- `internal/tigerfs/fuse/control_files.go`
- `internal/tigerfs/logging/logging.go`
- `internal/tigerfs/mount/registry.go`
- `internal/tigerfs/nfs/handler.go`
- `internal/tigerfs/nfs/mount_darwin.go`
- `README.md`
- `scripts/demo/demo.sh`
- `scripts/demo/init.sql`
- `scripts/demo/README.md`
- `scripts/demo/seed.sh`
- `scripts/install.sh`
- `scripts/pre-commit`
- `scripts/test-install.sh`
- `skills/tigerfs/data.md`
- `skills/tigerfs/files.md`
- `skills/tigerfs/ops.md`
- `skills/tigerfs/recipes.md`
- `skills/tigerfs/SKILL.md`
- `test/integration/crud_test.go`
- `test/integration/ddl_test.go`
- `test/integration/demo_data.go`
- `test/integration/format_test.go`
- `test/integration/fs_operations_test.go`
- `test/integration/main_test.go`
- `test/integration/migrate_test.go`
- `test/integration/pipeline_test.go`
- `test/integration/setup.go`
- `test/integration/synthesized_test.go`
- `test/integration/undo_test.go`
- `test/stress/README.md`
---
## 01. Overview
> What TigerFS exposes (mount + dot-directories), file-first vs data-first entry paths, runtime assumptions (PostgreSQL, FUSE/NFS), and the first routes to read.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/01-overview.md
- Generated: 2026-06-03T07:53:57.161Z
### Source Files
- `README.md`
- `docs/spec.md`
- `cmd/tigerfs/main.go`
- `internal/tigerfs/cmd/root.go`
- `docs/quickstart.md`
---
title: "Overview"
description: "What TigerFS exposes (mount + dot-directories), file-first vs data-first entry paths, runtime assumptions (PostgreSQL, FUSE/NFS), and the first routes to read."
---
TigerFS is a Go CLI (`tigerfs`) that mounts a PostgreSQL database at a local path and serves all filesystem semantics through `internal/tigerfs/fs.Operations`, with Linux using FUSE (`go-fuse`) and macOS using an in-process NFS v3 server (`go-nfs`). Every path under the mount resolves to SQL against PostgreSQL; Unix tools and agent file APIs operate on tables, rows, columns, and dot-directory capabilities without a separate query language.
## Runtime assumptions
| Requirement | Detail |
|-------------|--------|
| Database | PostgreSQL 12+ (driver: `pgx/v5`) |
| Linux | FUSE; user typically in `fuse` group |
| macOS | Native NFS mount to local NFS server (no FUSE kernel extension) |
| Connection | `postgres://` URI, `PG*` env vars, or `tiger:` / `ghost:` cloud prefixes |
| TLS | Non-localhost connections enforce `sslmode=require` unless `--insecure-no-ssl` |
| Go | 1.23+ to build from source |
The `mount` command connects once, registers the mount in `internal/tigerfs/mount`, and blocks until unmount or `SIGINT`/`SIGTERM`. `cmd/tigerfs/main.go` wires signal-aware shutdown into `cmd.Execute`.
Windows is listed in `docs/spec.md` as WinFsp-backed; Linux and macOS are the primary supported paths in the mount implementation (`mount_linux.go`, `mount_darwin.go`).
## What a mount exposes
At the mount root, tables from the default schema (usually `public`) appear as top-level directories. Non-default schemas use `/mount///` or explicit `/mount/.schemas///`.
```text
/mnt/db/ # mount root (PathRoot)
├── users/ # data-first table
│ ├── .info/ # schema, columns, count
│ ├── .by/ .filter/ .order/ # pipeline capabilities
│ ├── 123/ # row-as-directory
│ └── 123.json # row-as-file (not always in ls)
├── notes/ # file-first workspace (.md files)
│ ├── hello.md
│ ├── .history/ .log/ .savepoint/ .undo/
│ └── tutorials/
├── .build/ # create workspaces (PathBuild)
├── .create/ .modify/ .delete/ # DDL staging
├── .tables/ # tigerfs-schema backing tables
├── .schemas/ # explicit schema listing
└── .refresh # metadata cache invalidation
```
`ls` hides dot-directories by default; `ls -a` reveals `.info/`, `.by/`, and other control paths. Reserved dot names (`.build`, `.filter`, `.history`, etc.) cannot be created as user data — attempts return `EACCES`. User dotfiles such as `.gitignore` are allowed inside file-first workspaces when not on the reserved list.
## Dot-directories as the control surface
Capabilities are navigable paths, not separate APIs. `fs/path.go` classifies each path (`PathType`) and accumulates query state in `FSContext` for pipeline segments.
| Dot path | Role |
|----------|------|
| `.info/` | Table metadata (`schema`, `columns`, `count`, `indexes`) |
| `.by/` | Index-based navigation |
| `.filter/`, `.order/`, `.columns/` | Pipeline query segments |
| `.first/`, `.last/`, `.sample/` | Pagination |
| `.export/`, `.import/` | Bulk read/write |
| `.build/` | Create synthesized workspace (`markdown`, `plaintext`, `history`, …) |
| `.format/` | Add synthesized view to an existing table |
| `.create/`, `.modify/`, `.delete/` | DDL staging with `.commit` / `.abort` |
| `.history/`, `.log/`, `.savepoint/`, `.undo/` | File-first versioning and undo |
| `.tables/` | Data-first access to `tigerfs` schema backing tables |
Example pipeline (one SQL query):
```bash
cat /mnt/db/orders/.by/customer_id/123/.order/created_at/.last/10/.export/json
```
`--max-pipeline-depth` (default 10) limits how deep chained capabilities appear in listings.
## File-first vs data-first entry paths
Both modes share one mount and one PostgreSQL database. Per-directory mode is inferred from layout:
| Signal | Mode | Typical entry |
|--------|------|----------------|
| `.md` / `.txt` workspace files | File-first | `echo "markdown,history" > /mnt/db/.build/notes` |
| `.info/` under a table directory | Data-first | `ls /mnt/db` then `cat /mnt/db/users/.info/schema` |
| New project | File-first | `.build/` with `markdown` or `plaintext` |
| Existing database | Data-first | `tigerfs mount postgres://… /mnt/db` |
**File-first** presents rows as domain files (markdown with YAML frontmatter, or plaintext). Workspaces opt into `history` for `.log/`, `.savepoint/`, and `.undo/`. Subdirectories (`mkdir`, `mv`) map to hierarchical state; kanban-style flows use directory moves. Backing storage lives in the `tigerfs` schema; inspect via `/mnt/db/.tables//`.
**Data-first** presents each row as a file (`123.json`) and/or directory (`123/email.txt`). Updates use PATCH semantics for structured formats. Index navigation, pipeline paths, and DDL staging apply at the table level.
You can add file-first views to an existing table without rebuilding data:
```bash
echo "markdown" > /mnt/db/posts/.format/markdown
```
## Architecture
```mermaid
flowchart LR
subgraph clients [Clients]
Unix[Unix tools]
Agents[Agent file APIs]
end
subgraph adapters [Platform adapters]
FUSE[fuse.MountOps Linux]
NFS[nfs.Mount macOS]
end
subgraph core [Shared core]
Ops[fs.Operations]
Path[fs/path.go PathType]
DB[db + format]
end
PG[(PostgreSQL)]
Unix --> FUSE
Unix --> NFS
Agents --> FUSE
Agents --> NFS
FUSE --> Ops
NFS --> Ops
Ops --> Path
Ops --> DB
DB --> PG
```
New filesystem behavior belongs in `internal/tigerfs/fs/`, not in FUSE/NFS adapters. Linux defaults to `fuse.MountOps` (shared `fs.Operations`); `--legacy-fuse` selects the older FUSE node tree. macOS always uses NFS because FUSE requires third-party kernel extensions.
On macOS NFS, `go-nfs` synthesizes Open/Write/Close per WRITE RPC; content commits on Close. `ReadFile` always queries PostgreSQL fresh; stat/path caches hold metadata only with write invalidation.
## CLI surface
Root command (`internal/tigerfs/cmd/root.go`) registers:
`mount`, `unmount`, `stop`, `status`, `info`, `list`, `create`, `fork`, `migrate`, `test-connection`, `config`, `version`
`mount` accepts:
```bash
tigerfs mount postgres://user@host:5432/mydb /mnt/db
tigerfs mount tiger: # optional auto mountpoint
tigerfs mount ghost: /mnt/db
```
Configuration loads from `~/.config/tigerfs/config.yaml` with `TIGERFS_*` overrides (see configuration reference). Persistent flags include `--log-level`, `--config-dir`, `--read-only`, `--user-id`, and `--auto-savepoint-interval`.
Bundled agent skills live under `skills/tigerfs/` (`SKILL.md`, `files.md`, `data.md`, `ops.md`, `recipes.md`). The install script (`scripts/install.sh`) copies them into detected agent skill directories or stages them under `~/.config/tigerfs/skills/tigerfs`.
## Consistency model (read this early)
PostgreSQL is the single source of truth across concurrent mounts:
- **Do not cache row content** — reads must reflect latest commits.
- **Cache metadata only** — directory listings, stat sizes, column lists (short TTL, invalidation on writes).
- **Pipeline paths need isolated cache keys** — e.g. `.export/json` vs `.filter/active/true/.export/json` must not share stat entries.
Violations surface as cross-mount staleness; new write paths must invalidate `statCache` and `pathCache` with the user schema key.
## Verification signals
After install or build:
```bash
go build -o bin/tigerfs ./cmd/tigerfs
tigerfs version
tigerfs mount postgres://localhost/mydb /mnt/db &
ls /mnt/db
tigerfs unmount /mnt/db
```
For a zero-setup trial, `scripts/demo/demo.sh start` mounts sample data (Docker or native macOS). Success looks like table directories (`users/`, `orders/`, …) and readable row files (`cat users/1.json`).
## First routes to read
Use this order to go from mount behavior to implementation detail:
Read `README.md` for file-first vs data-first examples, dot-directory tables, and cloud backend prefixes.
Follow `docs/quickstart.md` for mount, `.build/` workspaces, savepoints, and pipeline `cat` paths.
`docs/file-first.md` — workspaces, frontmatter, `.tables/`. `docs/data-first.md` — row formats, PATCH writes, DDL, pipelines.
`docs/spec.md` — complete hierarchy, formats, configuration, error mapping, NFS/FUSE notes.
`CLAUDE.md` — package layout (`fs/`, `fuse/`, `nfs/`, `db/`), pipeline parsing in `fs/path.go`, synth layer in `fs/synth/`.
`skills/tigerfs/SKILL.md` — mode detection rules and safe patterns for agents using Read/Write/Glob/Grep.
| Topic | Location |
|-------|----------|
| Path parsing & `PathType` | `internal/tigerfs/fs/path.go` |
| Filesystem operations | `internal/tigerfs/fs/operations.go` |
| Mount & backends | `internal/tigerfs/cmd/mount.go`, `cmd/mount_linux.go`, `cmd/mount_darwin.go` |
| Synth formats | `internal/tigerfs/fs/synth/` |
| Integration tests | `test/integration/` |
## Related pages
Install paths, platform prerequisites (FUSE, NFS), and verification.
First mount, `ls`/`cat` exploration, and demo workflows.
Mount hierarchy and how paths map to SQL.
Mode detection, `.build/` vs raw tables, `.tables/` backing schema.
Pipeline grammar and `PathType` resolution.
FUSE vs NFS adapters, NFS write/commit model, mount registry.
Cross-mount freshness, cache TTLs, invalidation rules.
All cobra commands, flags, and connection argument patterns.
---
## 02. Installation
> Install paths (curl installer, go install, releases), platform prerequisites (Linux FUSE, macOS NFS, Docker), and verification commands.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/02-installation.md
- Generated: 2026-06-03T07:54:10.200Z
### Source Files
- `scripts/install.sh`
- `.goreleaser.yaml`
- `go.mod`
- `docs/spec.md`
- `scripts/test-install.sh`
---
title: "Installation"
description: "Install paths (curl installer, go install, releases), platform prerequisites (Linux FUSE, macOS NFS, Docker), and verification commands."
---
TigerFS ships as a single static `tigerfs` binary built with GoReleaser (`CGO_ENABLED=0`) for Linux and macOS on `x86_64` and `arm64`. Release archives bundle the CLI, `skills/tigerfs/` agent instructions, and SHA256 checksums served from `https://install.tigerfs.io`. Mounting requires PostgreSQL reachability; the filesystem backend is FUSE on Linux and an in-process NFS server on macOS.
## Install paths
The curl installer downloads a versioned release archive, verifies its checksum, installs the binary, and optionally copies agent skills.
```bash
curl -fsSL https://install.tigerfs.io | sh
```
Pin a version or override install location:
```bash
VERSION=v0.7.0 curl -fsSL https://install.tigerfs.io | sh
INSTALL_DIR=/usr/local/bin curl -fsSL https://install.tigerfs.io | sh
```
| Variable | Default | Effect |
|----------|---------|--------|
| `BASE_URL` | `https://install.tigerfs.io` | CDN root for `latest.txt` and `releases/{version}/` |
| `VERSION` | Latest from `/latest.txt` | Release tag; normalized to `v*` prefix |
| `INSTALL_DIR` | `~/.local/bin` if `~/.local` exists, else `~/bin` | Binary destination |
| `TIGERFS_INTERACTIVE` | unset | Set to `1` to force the agent-skills menu in non-TTY contexts |
**Installer dependencies:** `curl` or `wget`, `tar`, and `sha256sum` or `shasum`.
**Download layout:** `{BASE_URL}/releases/{version}/tigerfs_{OS}_{arch}.tar.gz` plus a sibling `.sha256` file. Supported OS values are `Linux` and `Darwin`; architectures are `x86_64` and `arm64`.
On macOS, the installer clears the Gatekeeper quarantine attribute (`xattr -d com.apple.quarantine`) when present.
Build and install the latest module release into your Go binary path:
```bash
go install github.com/timescale/tigerfs/cmd/tigerfs@latest
```
Requires Go **1.25.1** or newer (see `go.mod`). Agent skills are **not** copied by `go install`; clone the repository or use the curl installer to get `skills/tigerfs/`.
Tagged releases (`v*.*.*`) trigger `.github/workflows/release.yml`, which runs GoReleaser and publishes:
- **GitHub Releases** — per-platform `.tar.gz` archives and checksums
- **S3** (`tigerfs-releases`) — same artifacts under `releases/{tag}/`, plus root `latest.txt` and `install.sh` on stable (non-prerelease) tags
Archive names follow `tigerfs_{Os}_{Arch}.tar.gz` (for example `tigerfs_Linux_x86_64.tar.gz`). Each archive contains the `tigerfs` binary, `README.md`, `docs/spec.md`, and `skills/tigerfs/*`.
Download manually, verify the `.sha256` sidecar, extract, and place `tigerfs` on your `PATH`.
For development or unreleased commits:
```bash
git clone https://github.com/timescale/tigerfs.git
cd tigerfs
go build -o bin/tigerfs ./cmd/tigerfs
```
Optional snapshot matching release builds:
```bash
goreleaser release --snapshot --clean
./dist/tigerfs_*/*/tigerfs version
```
Skills live at `skills/tigerfs/` in the repository tree.
### Agent skills (curl / release archives)
Release archives include `skills/tigerfs/` (`SKILL.md`, `files.md`, `data.md`, `ops.md`, `recipes.md`). After binary install, `scripts/install.sh`:
- **Interactive (TTY or `TIGERFS_INTERACTIVE=1`):** prompts to install skills into detected agents (Claude Code, Cursor, Codex CLI, Gemini CLI, Windsurf, Antigravity, Kiro) or skip
- **Non-interactive (`curl | sh`):** stages skills to `~/.config/tigerfs/skills/tigerfs/` with copy hints for detected agents
Upgrades replace the target `tigerfs` skill directory (`rm -rf` then `cp -r`) so stale skill files are removed.
Validate the installer locally:
```bash
./scripts/test-install.sh
```
## Platform prerequisites
TigerFS needs a running **PostgreSQL** database (local, remote, or Tiger/Ghost backends). Filesystem prerequisites depend on the host OS.
```mermaid
flowchart LR
subgraph clients["Client tools"]
LS["ls / cat / grep"]
end
subgraph tigerfs["tigerfs process"]
OPS["fs.Operations"]
end
subgraph backends["OS filesystem backend"]
FUSE["Linux: go-fuse"]
NFS["macOS: go-nfs in-process"]
end
PG[(PostgreSQL)]
LS --> FUSE & NFS
FUSE & NFS --> OPS --> PG
```
| Platform | Filesystem backend | Host prerequisites |
|----------|-------------------|-------------------|
| **Linux** | FUSE (`hanwen/go-fuse/v2`) | FUSE userspace support; `/dev/fuse` present. CI installs `fuse` / `libfuse-dev`. Unprivileged mounts typically require membership in the `fuse` group. |
| **macOS** | In-process NFS v3 (`go-nfs`) | No FUSE or kernel extension install. Uses built-in `mount_nfs`. |
| **Windows** | WinFsp (planned) | Documented in `docs/spec.md` as requiring WinFsp; **not** in the current curl installer OS matrix (`Linux` / `Darwin` only). |
| **Docker** | Linux FUSE inside container | Privileged container, `/dev/fuse` device, `SYS_ADMIN` capability for mount workflows |
Managed platforms without FUSE device access (for example AWS Fargate, Cloud Run, Kubernetes without privileged pods) cannot run TigerFS mounts as implemented today.
### Linux FUSE setup
On Debian/Ubuntu-family systems:
```bash
sudo apt-get update
sudo apt-get install -y fuse libfuse-dev
```
Fedora/RHEL-style:
```bash
sudo dnf install fuse3
```
Confirm the device node exists before mounting:
```bash
test -e /dev/fuse && echo "FUSE available"
```
Integration and stress Docker stacks use `privileged: true`, map `/dev/fuse`, and add `SYS_ADMIN` (see `test/docker/docker-compose.test.yml`, `test/stress/docker/docker-compose.yml`).
### macOS
No additional filesystem packages. TigerFS selects the NFS backend at compile time (`mount_darwin.go`). The curl installer strips quarantine on the installed binary.
### Docker
**Root `Dockerfile`** — multi-stage Alpine image with `fuse` runtime packages; intended as a container entrypoint for the CLI (`ENTRYPOINT ["/usr/local/bin/tigerfs"]`).
**Demo stack** (`scripts/demo/docker/`) — Ubuntu-based image with `fuse3`, PostgreSQL client tools, and a prebuilt `tigerfs` binary. Compose runs the `tigerfs` service `privileged: true` against a `timescale/timescaledb-ha:pg18` database.
Typical FUSE mount flags in container docs and spec:
```bash
docker run --rm -it \
--device /dev/fuse \
--cap-add SYS_ADMIN \
-v /path/to/mount:/mnt/db \
tigerfs mount postgres://user:pass@host:5432/db /mnt/db
```
Use `./scripts/demo/demo.sh start` for a guided demo (auto-detects macOS NFS vs Linux Docker FUSE). See the demo environment page for `demo.sh` commands and seed data.
Package managers (Homebrew, apt, yum) are listed in `docs/spec.md` as **Phase 2 (deferred)**. The supported distribution paths today are curl install, `go install`, GitHub/S3 releases, and source build.
## Verification
Run these after any install path before attempting a mount.
```bash
command -v tigerfs
tigerfs version
```
```bash
$ tigerfs version
TigerFS v0.7.0
Build time: 2026-...
Git commit: ...
Go version: go1.25.1
Platform: darwin/arm64
```
Verify credentials without mounting:
```bash
tigerfs test-connection postgres://localhost:5432/mydb
```
Or use `PGHOST`, `PGUSER`, `PGDATABASE`, etc., or Tiger Cloud `TIGER_SERVICE_ID` per your config. The command prints PostgreSQL version, current database/user, and accessible schema/table counts (30s timeout).
From a source checkout:
```bash
./scripts/test-install.sh
```
Exercises download, checksum verification, binary placement, and skills staging/copy paths against a `file://` mock CDN.
### PATH
If `tigerfs` is not found, ensure the install directory is on `PATH`:
```bash
export PATH="$HOME/.local/bin:$PATH" # default curl target when ~/.local exists
# or
export PATH="$HOME/bin:$PATH"
```
The curl installer prints this hint when the install dir is missing from `PATH`.
### Install script integration test (maintainers)
`scripts/test-install.sh` validates non-interactive staging, interactive agent selection, skip behavior, and upgrade cleanup of stale skill files. Requirements match `install.sh`: `sh`, `tar`, `curl` or `wget`, `sha256sum` or `shasum`.
## Release and CDN topology
```text
git tag vX.Y.Z
│
▼
.github/workflows/release.yml → GoReleaser (.goreleaser.yaml)
│ ├─ linux/darwin × amd64/arm64
│ ├─ tar.gz + checksums
│ ├─ GitHub Releases
│ └─ S3 tigerfs-releases/releases/{tag}/
▼
CloudFront install.tigerfs.io
├─ /install.sh (from scripts/install.sh on stable tags)
├─ /latest.txt (current version string)
└─ /releases/{tag}/tigerfs_{OS}_{arch}.tar.gz[.sha256]
```
GoReleaser embeds version metadata via ldflags (`Version`, `BuildTime`, `GitCommit` in `internal/tigerfs/cmd`). Stable releases also upload `latest.txt` and invalidate CloudFront paths `/latest.txt` and `/install.sh`.
## What to do next
After installation and `test-connection` succeed, mount a database and explore with standard Unix tools. Use `--debug` on the CLI for structured stderr logs when mounts fail with errno-only tool messages.
## Related pages
First mount with `postgres://` or `tiger:`, explore with `ls`/`cat`, and demo success signals.
Linux FUSE vs macOS NFS adapters, write/commit behavior, and mount registry.
`postgres://` URIs, TLS/`sslmode=require`, localhost exemptions, and pool settings.
Bundled `skills/tigerfs` pack, install-time staging, and agent directory layout.
`demo.sh` workflows, Docker vs macOS native, and seeded exploration data.
Source builds, `go test`, Docker FUSE integration tests, and release checklist.
---
## 03. Quickstart
> First successful mount (postgres:// or tiger:), explore with ls/cat, demo.sh workflows, and expected success signals.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/03-quickstart.md
- Generated: 2026-06-03T07:53:36.240Z
### Source Files
- `docs/quickstart.md`
- `scripts/demo/demo.sh`
- `scripts/demo/README.md`
- `internal/tigerfs/cmd/mount.go`
- `test/integration/setup.go`
---
title: "Quickstart"
description: "First successful mount (postgres:// or tiger:), explore with ls/cat, demo.sh workflows, and expected success signals."
---
`tigerfs mount` connects to PostgreSQL (direct URI, `PG*` environment variables, or `tiger:` / `ghost:` backend prefixes) and exposes the default schema as a directory tree at a mountpoint. A successful first mount returns table names from `ls`, row payloads from `cat`, and registry entries visible to `tigerfs status`.
## Prerequisites
| Requirement | Notes |
|-------------|-------|
| TigerFS binary | `curl -fsSL https://install.tigerfs.io \| sh`, `go install github.com/timescale/tigerfs/cmd/tigerfs@latest`, or build from source (`go build -o bin/tigerfs ./cmd/tigerfs`) |
| PostgreSQL | Reachable database with at least one user schema and tables |
| Linux | FUSE support (typically pre-installed) |
| macOS | No extra kernel extensions; TigerFS uses an in-process NFS v3 server |
| Demo only | Docker with Compose v2; macOS native demo also needs Go 1.21+ |
Remote connections require TLS (`sslmode=require`) unless you pass `--insecure-no-ssl` or set `InsecureNoSSL` in config. Localhost and Unix sockets default to `sslmode=disable` when no `sslmode` is in the URI.
## Fastest path: demo environment
From the repository root, `scripts/demo/demo.sh` starts PostgreSQL, mounts TigerFS, seeds file-first apps, and prints exploration commands.
```bash
cd scripts/demo
./demo.sh start # Darwin → --mac; Linux → --docker
./demo.sh status
./demo.sh shell # cwd = mount root
ls
cat users/1.json
./demo.sh stop
```
```bash
cd scripts/demo
./demo.sh start --docker
./demo.sh shell # inside container at /mnt/db
ls
cat users/1.json
cat products/1.json
ls orders/.first/10/
./demo.sh stop
```
```bash
cd scripts/demo
./demo.sh start --mac
ls /tmp/tigerfs-demo
cat /tmp/tigerfs-demo/users/1.json
cat /tmp/tigerfs-demo/blog/hello-world.md
./demo.sh stop
```
### Demo lifecycle
```text
demo.sh start
├─ docker compose up (postgres; + tigerfs container in --docker mode)
├─ wait_for_postgres (init.sql → data-first tables)
├─ tigerfs mount postgres://demo:demo@… [--insecure-no-ssl]
├─ seed.sh → .build/blog|docs|snippets + sample files
└─ print success + example commands
demo.sh stop
├─ unmount / kill tigerfs
└─ docker compose down
```
| Command | Action |
|---------|--------|
| `start` | Build/start containers, mount, run `seed.sh` |
| `stop` | Unmount TigerFS, tear down containers |
| `status` | Report container and mount state |
| `restart` | `stop` then `start` |
| `shell` | Interactive session at mount root (`/mnt/db` in Docker, `/tmp/tigerfs-demo` on macOS) |
Both PostgreSQL and TigerFS run in containers. Mount inside the `tigerfs` container at `/mnt/db`. Connection: `postgres://demo:demo@postgres:5432/demo`.
PostgreSQL in Docker on `localhost:5432`; TigerFS runs natively on the host at `/tmp/tigerfs-demo`. Connection: `postgres://demo:demo@localhost:5432/demo`.
Passes `--log-level debug` to the mount command.
### Demo seed data
**Data-first** (from `init.sql` at PostgreSQL startup): `users` (~1,000 rows), `categories` (10, TEXT PK), `products` (~200), `orders` (~8,000, UUIDv7 PK), views, and indexes for `.by/` navigation.
**File-first** (from `seed.sh` after mount):
| Path | Format | History |
|------|--------|---------|
| `blog/` | Markdown, 5 posts | Yes |
| `docs/` | Markdown, 4 pages | Yes |
| `snippets/` | Plain text, 3 files | Yes |
## Mount your own database
```bash
tigerfs test-connection postgres://user:password@localhost:5432/mydb
```
```text
Connected to PostgreSQL 16.x ...
Database: mydb
User: user
Schemas: N
Tables: M
```
```bash
mkdir -p /mnt/db
tigerfs mount postgres://user:password@localhost:5432/mydb /mnt/db &
```
Or with standard PostgreSQL environment variables (no URI argument):
```bash
export PGHOST=localhost PGPORT=5432 PGUSER=user PGPASSWORD=password PGDATABASE=mydb
tigerfs mount /mnt/db &
```
```bash
ls /mnt/db
ls -la /mnt/db/users
cat /mnt/db/users/1.json
cat /mnt/db/users/.info/schema
cat /mnt/db/users/.info/count
```
```bash
tigerfs unmount /mnt/db
```
### Connection argument patterns
`mount` accepts one or two positional arguments. Resolution order:
| Args | Interpretation |
|------|----------------|
| `CONN MOUNT` | Explicit connection and mountpoint |
| `tiger:ID` or `ghost:ID` | Backend prefix; mountpoint defaults to `/` (config default `/tmp`) |
| `MOUNT` only | Mountpoint; connection from `PG*` env or config |
```bash title="postgres:// URI"
tigerfs mount postgres://user:pass@host:5432/db /mnt/db &
```
```bash title="PG* environment"
export PGHOST=localhost PGDATABASE=mydb PGUSER=user PGPASSWORD=secret
tigerfs mount /mnt/db &
```
```bash title="Tiger Cloud (auto mountpoint)"
tiger auth login # prerequisite
tigerfs mount tiger:abcde12345
# → /tmp/abcde12345 by default
```
```bash title="Ghost"
ghost login
tigerfs mount ghost:fghij67890 /mnt/cloud &
```
Demo and local Docker Postgres use `--insecure-no-ssl` because the sample connection strings omit TLS. Do not use that flag against production hosts.
### Useful mount flags
| Flag | Purpose |
|------|---------|
| `--foreground` | Run in foreground (default daemonizes) |
| `--read-only` | Disallow writes |
| `--schema` | Override default PostgreSQL schema |
| `--insecure-no-ssl` | Allow non-TLS remote connections |
| `--log-level debug` | Verbose structured logging to stderr |
## Explore with ls and cat
Each table in the default schema appears as a directory. Row IDs are files; extensions select serialization.
| Path | Result |
|------|--------|
| `/mnt/db/` | Table directories |
| `/mnt/db/users/123` | Row as TSV (default) |
| `/mnt/db/users/123.json` | Row as JSON object |
| `/mnt/db/users/123/email` | Single column value |
| `/mnt/db/users/.info/schema` | `CREATE TABLE` DDL |
| `/mnt/db/users/.info/columns` | Column names, one per line |
| `/mnt/db/users/.info/count` | `SELECT COUNT(*)` |
| `/mnt/db/users/.first/100/` | First 100 rows (directory listing) |
| `/mnt/db/users/.by/email/alice@example.com/` | Index lookup |
```bash
ls /mnt/db
cat /mnt/db/users/1.json
```
```text
categories/ orders/ products/ users/
{"id":1,"name":"Alice Smith","first_name":"Alice","last_name":"Smith",...}
```
Dot-directories (`.info`, `.filter`, `.by`, `.export`, …) are hidden from plain `ls` but visible with `ls -a` or `ls -la`.
### Minimal file-first workspace
```bash
echo "markdown,history" > /mnt/db/.build/notes
cat > /mnt/db/notes/hello.md << 'EOF'
---
title: Hello
---
Body text.
EOF
ls /mnt/db/notes/
cat /mnt/db/notes/hello.md
```
Writes go through PostgreSQL transactions; concurrent readers on other mounts see updates immediately (no row-content caching).
## Expected success signals
Use these checks to confirm the mount is healthy before deeper workflows.
### Filesystem probes
| Check | Success signal |
|-------|----------------|
| `ls ` | Lists table (and workspace) directories without errno |
| `cat /users/1.json` | Valid JSON for an existing row |
| `cat /users/.info/count` | Non-empty integer string |
| `ls -a /notes` | Shows `.history/`, `.savepoint/`, `.undo/` when `history` was enabled |
### CLI registry and connectivity
```bash
tigerfs status
tigerfs status /mnt/db
tigerfs test-connection postgres://...
```
Columns: `MOUNTPOINT`, `DATABASE` (password-redacted), `PID`, `STATUS` (`active` / `stale`), `UPTIME`. Empty table means no registered mounts.
Prints PostgreSQL version, `current_database()`, `current_user`, and counts of accessible schemas and tables. Exit code 0 only when the pool connects and queries succeed.
### Demo launcher output
After `./demo.sh start`:
```text
==> PostgreSQL is ready
==> Connection string: postgres://demo:demo@...
==> Mounting TigerFS at ...
==> Seeding file-first apps...
==> TigerFS mounted successfully!
```
`./demo.sh status` on a healthy run:
| Mode | Containers | Mount |
|------|------------|-------|
| `--docker` | `RUNNING` | `MOUNTED` at `/mnt/db` (inside container) |
| `--mac` | Postgres running | `MOUNTED` at `/tmp/tigerfs-demo` |
Failure paths exit non-zero with `Failed to mount TigerFS` or `PostgreSQL init did not complete after 60 seconds`.
### Process and OS mount table
```bash
# macOS / Linux
mount | grep tigerfs-demo # macOS demo path
pgrep -f "tigerfs.*mnt/db" # tigerfs process
# Docker demo
docker compose -f scripts/demo/docker/docker-compose.yml exec tigerfs mount | grep /mnt/db
```
On successful `tigerfs mount`, the daemon logs `Filesystem mounted successfully` (warn-level in production, debug with `--log-level debug`). `tigerfs unmount` prints `Successfully unmounted `.
## demo.sh exploration cheat sheet
Inside the mount root (via `./demo.sh shell` or `cd /tmp/tigerfs-demo`):
```bash title="Data-first"
cat users/1.json
cat products/.by/category/electronics/.export/json
ls orders/.sample/10/
cat categories/.export/tsv
```
```bash title="File-first"
ls blog/
cat blog/hello-world.md
cat docs/getting-started/installation.md
cat snippets/todo.txt
ls docs/.history/getting-started/installation.md/
```
## Common first-mount failures
| Symptom | Likely cause | Mitigation |
|---------|--------------|------------|
| `connection failed` from `test-connection` | Wrong host, credentials, or `sslmode` | Fix URI; use `--insecure-no-ssl` only for known-local/demo DBs |
| `Failed to mount TigerFS` in demo | Postgres not ready, port conflict, stale mount | `./demo.sh stop`; on macOS check `lsof -i :5432` |
| Empty `ls` at mountpoint | Wrong `--schema`, empty database, or mount not active | `tigerfs status`; confirm tables exist in target schema |
| `tigerfs mount tiger:ID` errors | Tiger CLI not authenticated | `tiger auth login`; verify service ID |
| Mount already exists | Prior demo or manual mount | `./demo.sh stop` or `tigerfs unmount ` |
Structured hints for filesystem operations appear on stderr as JSON when TigerFS returns errno (for example invalid synth task numbers). Run mounts with `--log-level debug` when diagnosing.
## Related pages
Install paths, platform prerequisites (FUSE vs NFS), and verification before your first mount.
Full demo architecture, `init.sql` / `seed.sh` split, and Docker vs macOS troubleshooting.
Mount hierarchy, dot-directory control surface, and path-to-SQL mapping.
`tiger:` and `ghost:` prefixes, `create` / `fork`, and CLI credential requirements.
`postgres://` URIs, `PG*` variables, TLS rules, and pool settings.
Stale mounts, force unmount, and errno recovery.
---
## 04. Filesystem as API
> Mount hierarchy, schema/table/row/column path model, dot-directory control surface, and how fs.Operations maps paths to SQL.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/04-filesystem-as-api.md
- Generated: 2026-06-03T07:54:06.838Z
### Source Files
- `docs/spec.md`
- `internal/tigerfs/fs/path.go`
- `internal/tigerfs/fs/operations.go`
- `internal/tigerfs/fs/context.go`
- `README.md`
---
title: "Filesystem as API"
description: "Mount hierarchy, schema/table/row/column path model, dot-directory control surface, and how fs.Operations maps paths to SQL."
---
TigerFS exposes PostgreSQL through a mount point where `fs.Operations` parses every path into a `ParsedPath` and `FSContext`, then issues SQL via `internal/tigerfs/db` — FUSE (Linux) and in-process NFS (macOS) are thin adapters that delegate `ReadDir`, `Stat`, and `ReadFile` to that core without reimplementing query logic.
## Request flow
```mermaid
flowchart LR
subgraph clients [Clients]
Unix[Unix tools ls cat grep]
end
subgraph adapters [Platform adapters]
FUSE[fuse.OpsFS + FSAdapter]
NFS[nfs.OpsFilesystem]
end
subgraph core [Shared core internal/tigerfs/fs]
Ops[Operations]
Parse[parsePath → ParsePath]
Ctx[FSContext]
DB[db.Client pipeline + row queries]
end
PG[(PostgreSQL)]
Unix --> FUSE
Unix --> NFS
FUSE --> Ops
NFS --> Ops
Ops --> Parse
Parse --> Ctx
Ops --> DB
DB --> PG
```
`ReadFile` always queries PostgreSQL for row content. `Stat` and directory listings may use short-TTL metadata caches; see the consistency page for invalidation rules.
## Mount hierarchy
After `tigerfs mount `, the mount root lists tables and views from the connection’s current schema (typically `public`), plus global control directories.
| Path | Role |
|------|------|
| `/` | Default-schema tables/views plus `.build`, `.create`, `.info`, `.schemas`, `.tables`, `.views` |
| `//` | One PostgreSQL table (or synth workspace view) as a directory |
| `/.schemas//` | Explicit schema; non-default schemas also appear as `///` |
| `/.tables//` | Backing table in the `tigerfs` schema (data-first escape hatch for synth apps) |
**Schema flattening:** Tables in the default schema appear at the mount root (`/users/`). Other schemas use a prefix (`/analytics/reports/`) or `/.schemas/analytics/reports/`. Root-level `/.build/` creates synthesized apps; `/.build/` under `/.schemas//` is rejected to avoid `tigerfs` schema collisions.
**Root listing** (`readDirRoot`) adds control dirs first, then cached table names, then views (synth views are writable `0755`; plain views `0555`).
## Path model: schema → table → row → column
Paths are absolute, start with `/`, and are parsed syntactically by `ParsePath` then enriched by `(*Operations).parsePath` (schema resolution and file-first policy gates).
### PathType dispatch
`ParsedPath.Type` drives all operations. Common values:
| PathType | Example path | Meaning |
|----------|--------------|---------|
| `PathRoot` | `/` | Mount root |
| `PathTable` | `/orders` or `/orders/.filter/status/shipped/` | Table dir; pipeline state in `FSContext` |
| `PathRow` | `/orders/123` or `/orders/123.json` | Row directory or format file |
| `PathColumn` | `/orders/123/email` | Single column |
| `PathCapability` | `/orders/.by/customer_id/` | Mid-pipeline capability listing |
| `PathExport` | `/orders/.export/json` | Terminal bulk export |
| `PathInfo` | `/orders/.info/count` | Table metadata file |
| `PathBuild` | `/.build/blog` | Create synth workspace (write format name) |
| `PathHistory` | `/blog/.history/post.md/` | Version snapshots (file-first) |
| `PathDDL` | `/.create/mytable/sql` | DDL staging |
Hierarchical synth paths (e.g. `/blog/tutorials/intro.md/.history/`) use **scan-ahead**: non-dot segments before a known capability become `PrimaryKey` / `RawSubPath`, then capability parsing continues.
### Dual row representation
Every row is reachable two ways:
1. **Row directory** — `///` appears in `ls`; contains per-column files and hidden format files (`.json`, `.csv`, `.tsv`, `.yaml`).
2. **Row file** — `//.json` (etc.) is readable but **not** listed, keeping `ls` output small.
Column files use type extensions (`.txt`, `.json`, `.bin`) per table metadata; full-row writes use PATCH semantics for `.json`/`.yaml`/`.tsv`/`.csv`.
### Format detection
Extensions `.json`, `.csv`, `.tsv`, `.yaml` set `ParsedPath.Format` for serialization in `readRowFile` via `internal/tigerfs/format`.
## Dot-directory control surface
Dot-prefixed names are reserved **capability directories** (hidden from plain `ls`, visible with `ls -a`). User dotfiles (`.gitignore`, `.env`) are allowed in file-first workspaces when not in the reserved set.
Capabilities are centralized in `internal/tigerfs/fs/constants.go`:
| Group | Directories | Purpose |
|-------|-------------|---------|
| Navigation | `.by`, `.filter`, `.order`, `.first`, `.last`, `.sample`, `.all` | Filters, sort, pagination (pipeline) |
| Projection / I/O | `.columns`, `.export`, `.import` | Column select, bulk export/import |
| Metadata | `.info` | `count`, `ddl`, `schema`, `columns`, `indexes` |
| Schema / DDL | `.schemas`, `.create`, `.modify`, `.delete`, `.indexes`, `.views` | DDL staging and schema browsing |
| Synth / workspace | `.build`, `.format`, `.history`, `.tables` | Create/configure workspaces, history, backing tables |
| Undo / audit | `.log`, `.savepoint`, `.undo` | Operation log, bookmarks, preview/apply undo |
| Mount | `/.info/user` | Mount-level identity for log entries |
Creating files or directories whose names match reserved capability names returns `EACCES`. Hard links or renames into those names must enforce the same rule.
### File-first vs data-first surfaces
On **synth workspace** paths (markdown/plaintext views), `rejectDataFirstCapOnSynthWorkspace` blocks data-first pipeline capabilities (`.by`, `.filter`, `.export`, table DDL, etc.) at the workspace root. Allowed workspace controls include `.info`, `.history`, `.log`, `.savepoint`, `.undo`, `.format`.
Data-first access to the backing table uses **`/.tables//...`**, which parses with `Schema=tigerfs` and bypasses the gate.
## FSContext: path state → SQL parameters
`FSContext` is **immutable**: each capability segment returns a cloned context with updated fields:
- `Filters` — AND-combined from `.by///` (indexed) and `.filter///`
- `OrderBy` / `OrderDesc` — from `.order//` (no further filters after order)
- `Limit` / `LimitType` — `.first`, `.last`, `.sample`; nested limits set `PreviousLimit*` and `NeedsSubquery()`
- `Columns` — `.columns/col1,col2/` then only `.export/` allowed
- `IsTerminal` — after `.export/`
- `PipelineDepth` — incremented per stage; compared to `max_pipeline_depth` (default **10**, `0` = unlimited)
`AvailableCapabilities()` drives which capability subdirs appear in `readDirTable`. When depth is exceeded or `HideCapabilities` is set (`.log`/`.savepoint` in workspaces), listings hide further pipeline dirs to avoid recursive scanner blowups (`find`, `rm -rf`, agents).
`ToQueryParams()` copies context into `db.QueryParams` for `QueryRowsPipeline` / `QueryRowsWithDataPipeline`.
### Pipeline → SQL
`internal/tigerfs/db/pipeline.go` builds one SQL statement from `QueryParams`:
- Simple: `WHERE` + `ORDER BY` + `LIMIT`
- Nested pagination: subquery wrapper (e.g. `.first/100/.last/50/`)
- Identifiers quoted via `QuoteIdent` / `QuoteTable`
Example path and effective query shape:
```bash
cat /mnt/db/orders/.by/customer_id/123/.order/created_at/.last/10/.export/json
```
```sql
SELECT * FROM "public"."orders"
WHERE "customer_id" = $1
ORDER BY "created_at" DESC
LIMIT 10
```
(Exact SQL depends on PK columns, projection, and nested limits.)
## How Operations maps operations to SQL
| Operation | Entry | SQL path |
|-----------|-------|----------|
| List table / pipeline dir | `ReadDir` → `readDirTable` | `ListRows` or `QueryRowsPipeline(ToQueryParams())` |
| Read export | `ReadFile` → `readExportFile` | `QueryRowsWithDataPipeline` or `GetAllRows` |
| Read row | `readRowFile` | `GetRow` + format encode; synth → synthesized markdown/plaintext |
| Read column | `readColumnFile` | Column value query |
| Read `.info/*` | `readInfoFile` | Metadata queries (`count`, DDL, etc.) |
| Stat | `statWithParsed` | Metadata cache + size heuristics; export sizes from pipeline |
Default row listing cap: `dir_listing_limit` (default **1000**). Pipeline listings without an explicit limit inherit the same cap. Large tables may refuse bare `ls` on the table root; use `.first/`, `.sample/`, or index paths (see spec large-table section).
`parsePath` also:
1. Fills empty `Context.Schema` from `GetCurrentSchema()` once per mount (`sync.Once`).
2. Applies synth workspace policy before any listing/read.
## Configuration knobs
Max rows returned when listing a table without an explicit pipeline limit (default 1000).
Hide further capability directories after this many pipeline stages (default 10; 0 = unlimited).
Threshold for `.filter/` distinct-value listing behavior on large tables (default 100000).
CLI: `tigerfs mount --max-ls-rows`, `--max-pipeline-depth`, etc. bind into the same `config.Config` struct passed to `NewOperations`.
## Mental model (compact)
```text
/mount/
├── .build/ .create/ .schemas/ .tables/ .views/ .info/ # mount control
├── / # public (or default) table
│ ├── .by/ .filter/ .order/ … .export/ # pipeline (data-first)
│ ├── .info/ .indexes/ .modify/ .delete/ # metadata + DDL
│ └── / # row dir
│ ├── col.txt age .json …
│ └── .json # row file (not in ls)
└── // # non-default schema
```
Synth workspace `notes/` looks like files (`hello.md`) but rows live in `tigerfs` backing tables; `/.tables/notes/` exposes the relational shape.
## Verification
```bash
tigerfs mount postgres://localhost/mydb /mnt/db
ls /mnt/db
ls -a /mnt/db/notes/ # expect .history, .log, etc. on history-enabled workspaces
```
```bash
ls /mnt/db/orders/.by/customer_id/
ls /mnt/db/orders/.by/customer_id/123/.last/10/
cat /mnt/db/orders/.by/customer_id/123/.last/10/.export/json | head
```
```bash
ls /mnt/db/.tables/notes/ # data-first columns/rows for synth app "notes"
```
Run with `--debug` to see structured zap logs when paths return `EINVAL`/`ENOENT` (hints often include the `/.tables/...` alternative for blocked synth paths).
## Related pages
What TigerFS exposes at mount time, runtime assumptions, and where to start reading.
Pipeline grammar, chaining rules, and PathType resolution in depth.
Mode detection, `.build`/`.format`, and `.tables/` backing schema.
TSV, CSV, JSON, YAML encoding and PATCH write semantics.
FUSE vs NFS adapters and delegation to `fs.Operations`.
Fresh reads vs metadata cache TTLs and write invalidation.
---
## 05. File-first and data-first
> When a mount path is a workspace vs a table, .build/.format vs raw tables, .tables/ backing schema, and mode detection rules.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/05-file-first-and-data-first.md
- Generated: 2026-06-03T07:54:43.669Z
### Source Files
- `docs/file-first.md`
- `docs/data-first.md`
- `skills/tigerfs/SKILL.md`
- `internal/tigerfs/fs/synth/build.go`
- `internal/tigerfs/fs/build.go`
---
title: "File-first and data-first"
description: "When a mount path is a workspace vs a table, .build/.format vs raw tables, .tables/ backing schema, and mode detection rules."
---
TigerFS exposes every mounted PostgreSQL database as a directory tree where each top-level name is either a **file-first workspace** (synthesized view presenting rows as `.md` or `.txt` files) or a **data-first table** (primary-key rows as `.json`/`.tsv`/column files plus pipeline capability directories). Runtime routing in `fs.Operations` calls `getSynthViewInfo` on the resolved schema and table/view name; a non-nil result switches listing, read, and write to the synth layer, while `rejectDataFirstCapOnSynthWorkspace` blocks pipeline capabilities on workspace paths unless you use `/.tables//`, which always parses with `schema=tigerfs`.
## Two surfaces on one mount
| Surface | Typical path | What you see | Primary write path |
|---------|--------------|--------------|-------------------|
| File-first workspace | `/notes/`, `/posts_md/` | `.md` or `.txt` files; with history: `.history/`, `.log/`, `.savepoint/`, `.undo/` at workspace root | `cat`, editors, `mv`, `rm` on files |
| Data-first table | `/users/`, `/orders/` | Row PKs, `.info/`, `.by/`, `.filter/`, `.export/`, etc. | Row/column files, pipeline paths, DDL staging |
| Backing table (data-first on synth storage) | `/.tables/notes/` | Same as data-first; schema fixed to `tigerfs` | Direct row/column writes (bypasses history triggers) |
Quick heuristic from the bundled agent skill: if a directory lists `.md`/`.txt` (and optionally history control dirs), treat it as file-first; if it exposes `.info/` and row PKs without a synth view, treat it as data-first.
```text
mount/ (default schema, e.g. public)
├── .build/ write-only: create workspaces
├── .tables/ data-first → tigerfs schema tables
│ └── notes/ backing table for workspace "notes"
├── notes/ file-first workspace (view in public)
│ ├── hello.md
│ ├── .history/ .log/ ... (when history enabled)
├── users/ data-first table
│ ├── .info/ .by/ ...
│ └── 1.json 2/
└── posts/ data-first (original table)
└── .format/markdown write → creates posts_md view
posts_md/ file-first view on existing posts table
```
## How mode is detected
Detection is **per directory name**, not per mount. `loadSynthCache` scans views in the connection’s current schema and builds a `synth.ViewInfo` map used by `getSynthViewInfo`.
```mermaid
flowchart TD
subgraph parse [Path parsing]
P[ParsePath + resolveSchema]
P --> Q{Context.Schema == tigerfs?}
Q -->|yes via /.tables/| DF[Data-first table ops]
Q -->|no| R{getSynthViewInfo non-nil?}
end
subgraph detect [Synth detection per view name]
C[PostgreSQL COMMENT ON VIEW]
C -->|tigerfs:md / tigerfs:txt / ,history| S[Synth view]
C -->|missing| V[View suffix _md _txt _tasks ...]
V -->|no match| Col[Column pattern filename+body ...]
Col -->|FormatNative| N[Not a workspace]
end
R -->|yes| FF[File-first synth ops]
R -->|no| DF
detect --> R
```
### Detection precedence
| Priority | Signal | Result |
|----------|--------|--------|
| 1 | View comment from `.build/` or `.format/` | `tigerfs:md`, `tigerfs:txt`, optional `,history` |
| 2 | View name suffix | `_md` → markdown, `_txt` → plaintext, `_tasks` / `_todo` / `_items` → tasks (tasks format not served in listings yet) |
| 3 | Column patterns | `filename`+`body`+extra → markdown; `filename`+`body` only → plaintext |
| 4 | Default | `FormatNative` → data-first table/view |
History is flagged from the comment or by presence of `tigerfs._history`. `FormatTasks` views are skipped in the synth cache until implemented.
### Workspace vs backing table
| Object | PostgreSQL location | Mount path |
|--------|---------------------|------------|
| Backing table | `tigerfs.` (e.g. `tigerfs.notes`) | `/.tables/notes/` |
| Workspace view | `.` (same name as app) | `/notes/` |
| Synthesized view on existing table | `._md` or `_txt` | `/posts_md/` (table `posts` unchanged at `/posts/`) |
`.build/` creates **both**: backing table in `tigerfs`, view in the user’s default schema, triggers, optional history hypertables (`_history`, `_log`, `_savepoint`, metadata). `.format/` only adds a view + comment over an **existing** table in the user schema; the original table path stays data-first.
## Creating file-first workspaces (`.build/`)
`/.build/` is mount-root only; `/.schemas//.build/` is rejected (ADR-015). Listing returns no entries (write-only).
Write a feature string to a virtual file `/.build/`:
```bash
echo "markdown,history" > /mnt/db/.build/notes
echo "markdown" > /mnt/db/.build/notes
echo "plaintext" > /mnt/db/.build/snippets
echo "history" > /mnt/db/.build/notes # add history to existing app only
```
`ls /mnt/db/` should show directory `notes/` (writable `0755` for synth views). With history and TimescaleDB: `.history/`, `.log/`, `.savepoint/`, `.undo/` appear at workspace root only.
`ls /mnt/db/.tables/notes/` exposes the `tigerfs.notes` table with full data-first capabilities.
Parsed by `synth.ParseFeatureString`. Supported formats: `markdown` / `md`, `plaintext` / `txt` / `text`. Append `,history` for versioned undo (requires TimescaleDB extension). Standalone `history` adds history to an existing workspace without recreating format.
`history` on `.build/` fails with a clear error if the `timescaledb` extension is not installed. Native/tasks-only strings are rejected for new app creation.
Markdown workspaces created by `.build/` include `id`, `parent_id`, `filename`, `filetype`, `title`, `author`, `headers` (JSONB), `body`, `encoding`, and timestamp columns—see `GenerateMarkdownTableSQL` in the synth build package.
## File-first on an existing table (`.format/`)
For a data-first table already at `/posts/`, add a synthesized view without moving data:
```bash
echo ok > /mnt/db/posts/.format/markdown
# Creates view posts_md in the same schema (SELECT * FROM posts + COMMENT tigerfs:md)
ls /mnt/db/posts_md/
```
| Input | View name | Extension |
|-------|-----------|-----------|
| `.format/markdown` | `_md` | `.md` |
| `.format/txt` | `_txt` | `.txt` |
`ls /mnt/db/posts/.format/` lists available format names (`markdown`, `txt`). Write content to `.format/` is ignored; the format comes from the filename. Unsupported formats return `ErrInvalidPath` with a hint.
## `/.tables/` and the `tigerfs` schema
`parseTablesPath` maps `/.tables//...` to `FSContext{Schema: "tigerfs", Table: name}` and then normal segment processing (rows, `.by/`, `.export/`, etc.).
| Path | Behavior |
|------|----------|
| `/.tables/` | `PathTablesList` — lists tables in `tigerfs` schema |
| `/.tables/notes/` | Data-first access to backing storage |
| `/.tables/notes/.filter/...` | Full pipeline (not gated) |
Writes under `/.tables//` hit the backing table directly and **do not** run workspace history triggers. Use `/workspace/` for audited edits; reserve `/.tables/` for diagnostics or repairs.
Auxiliary tables (`notes_history`, `notes_log`, `notes_savepoint`, `notes_metadata`) live in `tigerfs` and are used internally; workspace-level `.log/` and `.savepoint/` paths rewrite context to these tables while preserving `OrigTableName` for the UI.
## Capabilities on file-first workspaces
`rejectDataFirstCapOnSynthWorkspace` blocks data-first pipeline capabilities when `getSynthViewInfo` matches and schema is not `tigerfs`:
**Blocked on `/notes/...`:** `.by/`, `.filter/`, `.order/`, `.columns/`, `.first/`, `.last/`, `.sample/`, `.export/`, `.import/`, table DDL (`.modify/`, `.delete/`, index DDL), and filtered `PathTable` pipeline states.
**Allowed on workspaces:** `.info/`, `.history/`, `.log/`, `.savepoint/`, `.undo/`, `.format/` (workspace control surfaces).
Blocked paths return `ErrInvalidPath` with hint: use `mount/.tables//...` for data-first access.
## Column roles (file-first rendering)
Roles are auto-detected by naming convention (first match wins)—no explicit mapping file yet:
| Role | Candidate column names |
|------|------------------------|
| Filename | `filename`, `name`, `title`, `slug` |
| Body | `body`, `content`, `description`, `text` |
| Timestamps | `modified_at` / `updated_at`; `created_at` |
| Extra frontmatter (JSONB) | `headers` |
| Directories | `filetype` (`file` / `directory`) + `parent_id` |
Markdown frontmatter merges known columns plus `headers` keys (full replace on write for keys stored only in JSONB). Timestamp columns drive `ls -l` times but are not emitted in frontmatter.
## `.build/` vs `.format/`
| | `.build/` | `/.format/` |
|--|----------------|-------------------------|
| Target | New app name | Existing table |
| Storage | New table in `tigerfs` + view in default schema | View only (`_md` / `_txt`) |
| Schema design | Opinionated markdown/plaintext schema | Must already fit column conventions |
| History | Optional at create; add later with `echo history > .build/` | Not added by `.format/` alone |
| Default schema only | Yes (`/.build/` at mount root) | Yes (under table in resolved schema) |
| Listing | Write-only (empty `ls`) | Lists `markdown`, `txt` |
## Root mount layout
`readDirRoot` always includes: `.build`, `.create`, `.info`, `.schemas`, `.tables`, `.views`, then tables and views from `current_schema()`. Synth views are listed with mode `0755`; other views use `0555`. Legacy `_` backing tables in the user schema (pre-migration) trigger a one-time warning to run `tigerfs migrate`.
## Related pages
Path model, dot-directories, and how `fs.Operations` maps paths to SQL.
Markdown/plaintext I/O, directories, frontmatter, and workspace control files.
Row/column access, indexes, pagination, PATCH writes, and pipelines.
Backing tables, views, comments, and synth implementation details.
Pipeline grammar (`.by`, `.filter`, `.export`, …) used on data-first paths and `/.tables/`.
Versioned workspaces, `.log/`, savepoints, and undo apply paths.
---
## 06. Capability directories
> Pipeline path grammar (.by, .filter, .order, .first, .last, .export), chaining limits, SQL pushdown, and PathType resolution in path.go.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/06-capability-directories.md
- Generated: 2026-06-03T07:55:04.097Z
### Source Files
- `internal/tigerfs/fs/path.go`
- `internal/tigerfs/fs/constants.go`
- `internal/tigerfs/db/pipeline.go`
- `docs/data-first.md`
- `docs/spec.md`
- `test/integration/pipeline_test.go`
---
title: "Capability directories"
description: "Pipeline path grammar (.by, .filter, .order, .first, .last, .export), chaining limits, SQL pushdown, and PathType resolution in path.go."
---
Capability directories are dot-prefixed path segments under a table (for example `orders/.filter/status/pending/.last/10/`) that accumulate query state in an immutable `FSContext`. `ParsePath` in `internal/tigerfs/fs/path.go` performs syntactic parsing only; `Operations.parsePath` resolves schema, applies file-first policy gates, and hands the resulting `ParsedPath` to `readDirWithParsed` / `readFileWithParsed`. When `FSContext.HasPipelineOperations()` is true, row listings and `.export/` reads call `db.QueryRowsPipeline` or `db.QueryRowsWithDataPipeline`, which compile the full chain into one parameterized SQL statement.
## Pipeline capabilities
Navigation and bulk-query capabilities are defined as constants in `internal/tigerfs/fs/constants.go`:
| Directory | Segments | Effect on `FSContext` |
|-----------|----------|------------------------|
| `.by/` | `.by/`, `.by//`, `.by///` | Equality filter; `Indexed: true` on the filter |
| `.filter/` | `.filter/`, `.filter//`, `.filter///` | Equality filter; `Indexed: false` |
| `.order/` | `.order/`, `.order//`, `.order/.desc/` | Sets `OrderBy` / `OrderDesc`; blocks further filters |
| `.columns/` | `.columns/`, `.columns/col1,col2/` | Column projection; only `.export/` allowed next |
| `.first/N/` | Positive integer `N` | `LimitFirst` |
| `.last/N/` | Positive integer `N` | `LimitLast` |
| `.sample/N/` | Positive integer `N` | `LimitSample` |
| `.export/` | `.export/`, `.export/.with-headers/` | Terminal; sets `IsTerminal` → `PathExport` |
| `.all/` | `.all/` | No-op limit (equivalent to table root; hidden from `ls`) |
Supported export format names: `csv`, `tsv`, `json`, `yaml` (with or without a file extension such as `data.csv`).
`.import/`, `.info/`, DDL (`.create/`, `.modify/`, `.delete/`), `.indexes/`, and synth/history paths (`.build/`, `.format/`, `.history/`, `.log/`, `.savepoint/`, `.undo/`) are separate capability families. This page focuses on the data-first **pipeline** set: `.by`, `.filter`, `.order`, `.columns`, `.first`, `.last`, `.sample`, `.export`.
### Example paths
```bash
# List pending orders for customer 123, newest first, as JSON
cat /mnt/db/orders/.by/customer_id/123/.filter/status/pending/.order/created_at/.last/10/.export/json
# Project columns then export
cat /mnt/db/orders/.filter/status/shipped/.columns/id,total,created_at/.export/csv
# Nested pagination: last 50 rows among the first 100 by PK
ls /mnt/db/events/.first/100/.last/50/
```
## Path grammar and `PathType` resolution
`ParsePath` splits the mount path into segments and routes through `parseTablePath` → `processSegments`. Known capabilities are recognized by `isKnownCapability`; unknown dot-segments (for example `.git`) are treated as ordinary filenames via scan-ahead logic.
```mermaid
flowchart TD
subgraph parse ["fs/path.go — ParsePath"]
PP[ParsePath]
PT[parseTablePath]
PS[processSegments]
PC[processCapability]
PR[processRowOrColumn]
end
subgraph types ["ParsedPath.Type"]
PCap[PathCapability — incomplete cap]
PTbl[PathTable — filters/limits/order applied]
PExp[PathExport — terminal export]
PRow[PathRow / PathColumn]
end
subgraph runtime ["fs/operations.go"]
OP[Operations.parsePath]
RD[readDirWithParsed / readFileWithParsed]
QP[FSContext.ToQueryParams]
DB[db.QueryRowsPipeline]
end
PP --> PT --> PS
PS -->|known cap| PC
PS -->|no cap ahead| PR
PC -->|listing only| PCap
PC -->|filter/limit/order/columns| PTbl
PC -->|.export/| PExp
PR --> PRow
OP --> PP
PTbl --> RD
PExp --> RD
PTbl --> QP --> DB
```
### How `PathType` is chosen
| Path shape | `PathType` | Notes |
|------------|------------|-------|
| `/users` | `PathTable` | Empty `FSContext`; schema filled at runtime |
| `/users/.by/email` | `PathCapability` | `CapabilityDir=.by`, `CapabilityArg=email` |
| `/users/.by/email/foo@bar.com` | `PathTable` | Filter applied; type collapses from `PathCapability` |
| `/users/.first/10` | `PathTable` | Limit applied |
| `/users/.export/json` | `PathExport` | `IsTerminal=true`; no further caps |
| `/users/123/name` | `PathColumn` | Terminal row/column segment |
| `/users/.filter/active/.order/date/.last/10` | `PathTable` | Full pipeline in `Context` |
Incomplete capabilities (directory listing forms) keep `PathCapability` and set `CapabilityDir` / `CapabilityArg`. Once a filter value, limit, order, or column list is applied, `processBy` / `processFilter` / `processLimit` / `processOrder` / `processColumns` reset the type to `PathTable` while mutating `Context`.
`processRowOrColumn` runs only when no known capability appears ahead in the remaining segments. Hierarchical synth paths use scan-ahead: segments before `.history/` become `PrimaryKey` / `RawSubPath`, then capability processing continues.
### Reserved names
`IsCapabilityDirectory` in `constants.go` prevents names like `.filter` from being interpreted as column values during row/column parsing. The reserved set includes pipeline dirs plus `.info`, `.log`, `.savepoint`, and `.undo`.
## `FSContext` and chaining rules
`FSContext` (`internal/tigerfs/fs/context.go`) is immutable: each pipeline step returns a clone via `WithFilter`, `WithOrder`, `WithLimit`, `WithColumns`, or `WithTerminal`. `PipelineDepth` increments on each mutating step and drives `max_pipeline_depth` listing suppression.
Enforcement methods:
| Method | Rule |
|--------|------|
| `CanAddFilter()` | False after `.order/` or `.export/` |
| `CanAddOrder()` | False if already ordered (second `.order/` rejected at parse time) |
| `CanAddLimit(type)` | No `.first` after `.first`, no `.last` after `.last`, no limits after `.sample/` |
| `CanAddColumns()` | False after `.columns/` or `.export/` |
| `CanExport()` | False when `IsTerminal` |
`AvailableCapabilities()` drives which capability directories appear in `ls` after pipeline steps. After `.columns/`, only `.export/` is advertised.
### Allowed chaining (parent → children)
| Parent state | Next capabilities |
|--------------|-------------------|
| Table root | `.by`, `.filter`, `.order`, `.columns`, `.first`, `.last`, `.sample`, `.export` |
| After `.by///` or `.filter///` | Same as root (filters AND-combined) |
| After `.order//` | `.columns`, `.first`, `.last`, `.sample`, `.export` (no more filters) |
| After `.columns/col1,col2/` | `.export` only |
| After `.first/N/` or `.last/N/` | Filters, order, columns, other limit types, `.export` |
| After `.sample/N/` | Filters, order, columns, `.export` (no further limits) |
| After `.export/` | None (terminal) |
### Disallowed combinations (parse-time errors)
| Pattern | Replacement / behavior |
|---------|------------------------|
| `.first/N/.first/M/` | Use single `.first/M/` |
| `.last/N/.last/M/` | Use single `.last/M/` |
| `.sample/N/.first|M` or `.sample/N/.last/M` | Use `.sample/M/` |
| `.order/a/.order/b/` | Rejected; only one order per path |
| `.columns/a,b/.columns/c,d/` | Rejected; merge column lists |
| `.by/` or `.filter/` after `.order/` | Rejected with hint: filters before order |
| `.export/.../.first/` | Export is terminal |
Post-limit filters are allowed: `.first/100/.filter/status/active/` applies the filter to the subquery result set.
### Nested pagination
When a second limit is added, `WithLimit` moves the current limit to `PreviousLimit` / `PreviousLimitType`. `FSContext.NeedsSubquery()` becomes true, and `db.buildNestedPipelineSQL` wraps the inner limit in a subquery before applying outer filters, order, and limit.
| Path | Meaning |
|------|---------|
| `.first/100/.last/50/` | Last 50 rows of the first 100 by PK (rows 51–100 of PK-ordered set) |
| `.last/100/.first/50/` | First 50 of the last 100 |
| `.first/1000/.sample/50/` | Random 50 drawn from the first 1000 |
Integration tests in `test/integration/pipeline_test.go` (`TestPipeline_NestedPagination`) verify `.first/10/.last/3/` and `.last/10/.first/3/` mount listings.
## SQL pushdown
At runtime, `readDirTable` checks `fsCtx.HasPipelineOperations()`. When true, it builds `params := fsCtx.ToQueryParams()`, fills `PKColumns` from metadata, applies `DirListingLimit` if no explicit limit was set, and calls `db.QueryRowsPipeline`. Export reads use `QueryRowsWithDataPipeline` with `selectPKOnly=false` so column projection and filters apply to the full row payload.
`db.QueryParams` (`internal/tigerfs/db/pipeline.go`) carries:
- `Filters` — AND-combined equalities (`column = $n`)
- `OrderBy` / `OrderDesc` — custom sort with PK tie-breakers
- `Limit` / `LimitType` — `LIMIT` plus default `ORDER BY` (PK ASC for `.first/`, PK DESC for `.last/`, `RANDOM()` for `.sample/`)
- `PreviousLimit` / `PreviousLimitType` — nested subquery when needed
- `Columns` — quoted column list instead of `SELECT *`
`FilterCondition.Indexed` (from `.by/` vs `.filter/`) does not change SQL semantics; both emit equality predicates. The flag is reserved for planning hints.
Simple (non-nested) pipeline SQL shape:
```sql
SELECT FROM "schema"."table"
WHERE col1 = $1 AND col2 = $2
ORDER BY ASC NULLS LAST,
LIMIT $n
```
Nested `.first/100/.last/50/` compiles to an inner `SELECT * ... ORDER BY pk ASC LIMIT 100` wrapped by an outer query with `ORDER BY pk DESC LIMIT 50`.
`.sample/` uses `ORDER BY RANDOM()` and can be expensive on large tables. Prefer indexed `.by/` paths and explicit `.first/` / `.last/` windows for predictable cost.
## `.by/` vs `.filter/`
Both add equality filters to `FSContext`. The difference is navigation and listing behavior, not the final filter SQL.
| Aspect | `.by/` | `.filter/` |
|--------|--------|------------|
| Column listing (`ls`) | Indexed columns only (`GetSingleColumnIndexes`) | All table columns |
| Value listing (`ls .by/col/`) | `DirListingLimit` (default 1000) | `DirFilterLimit` (default 100000); may surface `.table-too-large` |
| Filter flag | `Indexed: true` | `Indexed: false` |
| Typical use | Index-backed lookups | Ad-hoc filters on non-indexed columns |
Direct access without listing still works when value listing is blocked:
```bash
cat /mnt/db/large_events/.filter/type/click/.first/100/.export/json
```
On `tigerfs._log` tables, `.by/filename/` and `.filter/filename/` are rejected at parse time (`blockLogFilenameQuery`): log `filename` values contain `/`, which cannot appear in a single path segment. Use `.log/.by/file_id//` instead.
## Directory listings vs reads
| Operation | Pipeline behavior |
|-----------|-------------------|
| `ls` on `PathTable` with pipeline context | `QueryRowsPipeline` for row PKs; capability subdirs from `AvailableCapabilities()` |
| `ls` on `PathCapability` | Column/value/order listing helpers (`.by/`, `.filter/`, `.columns/`, `.order/`) |
| `cat` on `PathExport` | `QueryRowsWithDataPipeline` + format encoder |
| `cat` on row files under a pipeline path | Row read respects accumulated filters (via parsed context) |
Pagination directories (`.first/`, `.last/`, `.sample/`) appear in listings but show empty contents until you navigate to a numeric child (for example `.first/50/`). This avoids recursive tools (`find`, `rm -rf`, agents) walking infinite placeholder trees.
### `max_pipeline_depth`
Maximum chained pipeline operations before capability directories are hidden from `ls`. `0` means unlimited depth for listings. Parsing and explicit paths still work when depth is exceeded; only directory advertisements are suppressed. Default comes from `config.Config` / `~/.config/tigerfs/config.yaml`.
When `fsCtx.PipelineDepth >= max_pipeline_depth`, `readDirTable` lists rows and `.info/` only—no further `.by/`, `.filter/`, etc. `.log/` and `.savepoint/` under file-first workspaces set `HideCapabilities` for the same recursion protection; pipeline segments remain reachable by full path.
## File-first workspace boundary
`Operations.rejectDataFirstCapOnSynthWorkspace` blocks `.by/`, `.filter/`, `.order/`, `.columns/`, `.first/`, `.last/`, `.sample/`, `.export/`, and related DDL on synth workspace mount paths (`//...`). Data-first pipeline access to the backing table is available through `/.tables//...` (schema `tigerfs`). Allowed workspace controls include `.info/`, `.history/`, `.log/`, `.savepoint/`, `.undo/`, and `.format/`.
## Configuration and errors
| Setting | Default | Role |
|---------|---------|------|
| `max_pipeline_depth` | `10` | Hide capability dirs in deep chains |
| `dir_listing_limit` | `1000` | Default row cap for pipeline `ls` when no `.first/` / `.last/` / `.sample/` |
| `dir_filter_limit` | `100000` | Threshold for `.filter//` value listing |
Invalid chains return `ErrInvalidPath` from `ParsePath` with a `Hint` field (surfaced via structured logging and POSIX `EINVAL` on FUSE/NFS). Examples: `"cannot add .filter/ after .order/"`, `"cannot add .first after .first"`.
## Implementation map
| Layer | Responsibility |
|-------|----------------|
| `fs/path.go` | `ParsePath`, `processCapability`, `PathType`, capability-specific parsers |
| `fs/context.go` | `FSContext`, chaining rules, `ToQueryParams()` |
| `fs/constants.go` | Directory name constants, `IsCapabilityDirectory` |
| `fs/operations.go` | `parsePath`, `readDirTable`, capability listing, synth/file-first gates |
| `db/pipeline.go` | SQL generation, `QueryRowsPipeline`, nested subqueries |
Production code should call `Operations.parsePath`, not bare `ParsePath`, so schema resolution and workspace policy apply before any database access.
## Related pages
Mount hierarchy, dot-directory control surface, and how operations map paths to SQL.
Row/column reads, index navigation, pagination, PATCH writes, and export/import workflows.
No content cache on reads; stat/path cache keys must include full pipeline paths.
`max_pipeline_depth`, `dir_listing_limit`, `dir_filter_limit`, and mount flags.
---
## 07. Synthesized apps
> Workspace creation via .build/, markdown/plaintext formats, backing tables in tigerfs schema, views, and column/frontmatter mapping.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/07-synthesized-apps.md
- Generated: 2026-06-03T07:55:30.250Z
### Source Files
- `docs/file-first.md`
- `internal/tigerfs/fs/synth/markdown.go`
- `internal/tigerfs/fs/synth/plaintext.go`
- `internal/tigerfs/fs/synth/build.go`
- `internal/tigerfs/fs/synth/format.go`
- `test/integration/synthesized_test.go`
---
title: "Synthesized apps"
description: "Workspace creation via .build/, markdown/plaintext formats, backing tables in tigerfs schema, views, and column/frontmatter mapping."
---
TigerFS **synthesized apps** expose PostgreSQL views as directories of real files (`.md` with YAML frontmatter, or plain `.txt` bodies). Writes to `/.build/` or `//.format/` create the backing table, view, triggers, and `COMMENT ON VIEW` markers; `fs.Operations` in `internal/tigerfs/fs/synth_ops.go` maps each file path to `INSERT`/`UPDATE`/`DELETE` on the view while `internal/tigerfs/fs/synth/` handles synthesis and parsing.
## Creation paths
Two filesystem entry points provision synthesized apps. They differ in whether TigerFS also creates the backing table and in how the mount path is named.
| Path | Write target | Creates | Mount path after success |
|------|--------------|---------|--------------------------|
| **`.build/`** | `/.build/` | `tigerfs.` table, `` view in the connection schema, triggers, optional history | `//` (same name as app) |
| **`.format/`** | `//.format/` | Synthesized view on an **existing** table only | `/_md/` or `/_txt/` |
Write a comma-separated feature string to a virtual file under `/.build/`:
```bash
echo "markdown,history" > /mnt/db/.build/notes
echo "markdown" > /mnt/db/.build/posts
echo "plaintext" > /mnt/db/.build/snippets
echo "history" > /mnt/db/.build/notes # add history to an existing .build/ app
```
Feature string parsed by `synth.ParseFeatureString`: `markdown` / `md`, `txt` / `text` / `plaintext` / `plain`, optional `,history`.
`.build/` is **write-only** at the mount root (listing returns no entries). `.build/` under `/.schemas//` is rejected — use the default schema mount.
```bash
echo ok > /mnt/db/posts/.format/markdown
echo ok > /mnt/db/logs/.format/txt
```
The format name is taken from the **filename** (`markdown`, `txt`); file body content is ignored. `synth.AvailableFormats()` currently exposes `markdown` and `txt` only.
```bash
ls /mnt/db/ # expect or _md
ls /mnt/db/.tables/ # expect backing tables in tigerfs schema
cat /mnt/db/notes/hello.md # synthesized read
```
`history` in a `.build/` string requires the **TimescaleDB** extension. Without it, TigerFS returns an error with a hint to install TimescaleDB.
## Storage layout
`.build/` apps split storage across two schemas:
```text
User schema (e.g. public) tigerfs schema
--------------------------- ----------------
VIEW notes ──SELECT *──► TABLE notes
COMMENT 'tigerfs:md,history' TABLE notes_history (if history)
TABLE notes_log
TABLE notes_savepoint
TABLE notes_metadata
FUNCTION resolve_path(...)
```
- **Backing table:** `tigerfs.` — UUIDv7 `id`, `parent_id`, `filename`, `filetype` (`file` | `directory`), format-specific columns, `encoding` (`utf8` | `base64`), timestamps.
- **Synthesized view:** `` in the user schema — `CREATE VIEW … AS SELECT * FROM tigerfs.` plus `COMMENT ON VIEW` (`tigerfs:md`, `tigerfs:txt`, or with `,history`).
- **`.format/` views:** `_md` or `_txt` in the user schema; the original `/` path stays data-first.
Access the backing table without synthesis:
```bash
ls /mnt/db/.tables/notes/ # row/column layout on tigerfs.notes
ls /mnt/db/notes/ # file-first synthesized view
```
Capability directories (`.filter`, `.export`, etc.) work under `/.tables//` but are blocked on the synthesized workspace path — use `.tables/` for pipeline operations on backing data.
## Supported formats
| Format | `.build/` token | View comment | File shape | Status |
|--------|-----------------|--------------|------------|--------|
| Markdown | `markdown`, `md` | `tigerfs:md` | YAML frontmatter + body | Fully implemented |
| Plain text | `txt`, `plaintext`, `text`, `plain` | `tigerfs:txt` | Body only | Fully implemented |
| Tasks | — (not in `.build/`) | `tigerfs:tasks` | Numbered task filenames | Detected in `synth.DetectFormat`; **skipped** in synth cache (`FormatTasks` not loaded) |
TigerFS does **not** auto-append extensions. Filenames come from the `filename` column exactly as stored (e.g. `hello-world.md` vs `hello-world`).
### Markdown backing schema (`.build/`)
`GenerateMarkdownTableSQL` creates:
| Column | Role |
|--------|------|
| `id` | UUID primary key (uuidv7) |
| `parent_id` | Directory hierarchy (self-FK) |
| `filename` | Display path / leaf name |
| `filetype` | `file` or `directory` |
| `title`, `author` | Example dedicated frontmatter columns |
| `headers` | JSONB for keys without dedicated columns |
| `body` | Markdown body |
| `encoding` | `utf8` or `base64` |
| `created_at`, `modified_at` | File times (not rendered in frontmatter) |
### Plain text backing schema (`.build/`)
Same hierarchy columns; `body` only (no `title`/`author`/`headers`).
## Format detection
At mount, `Operations.loadSynthCache` builds a per-schema map of `synth.ViewInfo`:
1. **View comment** — `tigerfs:md`, `tigerfs:txt`, optional `,history` (set by `.build/` / `.format/`).
2. **View name suffix** — `_md`, `_txt`, `_tasks`, `_todo`, `_items`.
3. **Column patterns** — `filename`+`body`+others → markdown; `filename`+`body` only → plain text; `number`+`name`+`status`+`body` → tasks.
Suffix and comment take precedence over column inference.
## Column and frontmatter mapping
`DetectColumnRoles` assigns columns by **naming convention** (first match wins). Explicit per-column mapping is documented as planned in `docs/file-first.md` but not implemented in code.
| Role | Convention names (priority order) |
|------|-----------------------------------|
| Filename | `filename`, `name`, `title`, `slug` |
| Body | `body`, `content`, `description`, `text` |
| Modified / created time | `modified_at`, `updated_at` / `created_at` |
| Extra headers (JSONB) | `headers` |
| Hierarchy | `filetype`, `parent_id` |
| Encoding | `encoding` |
| Frontmatter (markdown) | All other columns except PK and roles above |
**Read path:** `SynthesizeMarkdown` emits `---` YAML from frontmatter columns (schema order), then merges `headers` JSONB keys alphabetically, then the body. `SynthesizePlainText` returns body only.
**Write path:** `ParseMarkdown` → `MapToColumns` → partial `UpdateRow` (only parsed columns are SET).
| Key type | Write behavior |
|----------|----------------|
| Known frontmatter column | Omitted in YAML → column not in UPDATE → **previous value kept** |
| `headers` JSONB | **Full replace** from keys present in YAML; omitted keys are removed |
| Body | Always replaced with file content |
| Timestamps | Driven by DB defaults/triggers; not set from frontmatter |
Clear a known column explicitly: `title: ""`. Unknown frontmatter keys without a `headers` column return a parse error listing valid keys.
Binary content (NUL or invalid UTF-8) is stored as base64 in `body` with `encoding = base64`; text writes set `encoding = utf8`.
## Directory hierarchy
`.build/` tables include `filetype` and `parent_id`, enabling POSIX-style trees:
```bash
mkdir /mnt/db/notes/tutorials
echo "..." > /mnt/db/notes/tutorials/getting-started.md
mv /mnt/db/notes/tutorials /mnt/db/notes/guides # atomic directory rename
```
- Parent directories are auto-created on write when missing.
- `tigerfs.resolve_path` resolves path segments for lookups.
- Parent directory `modified_at` is bumped on child create/delete/rename (not on content-only edits).
- Reserved names (`.history`, `.log`, `.savepoint`, `.undo`, capability dirs) cannot be used as user filenames; virtual dirs take precedence.
Views created only via `.format/` on tables **without** `filetype` cannot use `mkdir` on the synthesized path — recreate with `.build/` or add hierarchy columns manually.
## History-enabled workspaces
When `,history` is included (or `echo history > /.build/` on an existing app), TigerFS adds Timescale hypertables and triggers in `tigerfs`:
| Artifact | Table |
|----------|-------|
| Version archive | `_history` |
| Operation log | `_log` |
| Savepoints | `_savepoint` |
| Format boundaries | `_metadata` |
The workspace root exposes `.history/`, `.log/`, `.savepoint/`, and `.undo/` (read-only history; undo via `.apply`). Subdirectories do not duplicate these control dirs.
History format migrations record boundaries in `_metadata`; undo across those boundaries is refused. See the history page for savepoint and undo paths.
## Implementation map
```mermaid
flowchart TB
subgraph mount["Mount path"]
BUILD["/.build/name"]
FORMAT["/table/.format/markdown"]
FILES["/app/path/file.md"]
TABLES["/.tables/app/..."]
end
subgraph fsops["internal/tigerfs/fs"]
WBF["writeBuildFile"]
WFF["writeFormatFile"]
WSF["writeSynthFile"]
CACHE["loadSynthCache / ViewInfo"]
end
subgraph synthpkg["internal/tigerfs/fs/synth"]
GSQL["GenerateBuildSQLWithFeatures"]
SYN["SynthesizeMarkdown / PlainText"]
PAR["ParseMarkdown / MapToColumns"]
COL["DetectColumnRoles"]
end
subgraph pg["PostgreSQL"]
TBL["tigerfs.app"]
VIEW["schema.app or app_md"]
end
BUILD --> WBF --> GSQL --> TBL
GSQL --> VIEW
FORMAT --> WFF --> VIEW
FILES --> WSF --> CACHE
CACHE --> COL
WSF --> PAR --> VIEW
VIEW --> TBL
FILES --> SYN
SYN --> COL
TABLES --> TBL
```
After `.build/` or `.format/` writes, `invalidateSynthCache()` and metadata cache invalidation ensure new views are detected on the next access.
## Errors and constraints
| Situation | Result |
|-----------|--------|
| Unsupported `.build/` format (e.g. `tasks`) | `ErrInvalidPath` — supported: `markdown`, `txt`, optional `,history` |
| `history` without TimescaleDB | `ErrInvalidPath` with install hint |
| `.build/` without app name | Hint: `echo markdown > .build/posts` |
| Unknown frontmatter key (no `headers` column) | Parse error listing valid keys |
| `mkdir` on view without `filetype` | Error — use `.build/` or add columns |
| Filename collides with `.undo`, etc. | Reserved-name error |
Structured hints are logged to stderr (zap) before errno is returned to the caller.
## Related pages
Day-to-day read/write, directories, and frontmatter examples on mounted workspaces.
When a path is a workspace vs a table, and how `.tables/` relates to `.build/`.
`.history/`, `.log/`, `.savepoint/`, `.undo/` for history-enabled synthesized apps.
Mount hierarchy, dot-directories, and how `path.go` resolves paths to SQL.
Pipeline paths on `/.tables//` for filters, export, and pagination.
---
## 08. Consistency and caching
> Cross-mount read freshness (no content cache), metadata/stat/path cache TTLs, invalidation on writes, and pipeline cache key isolation.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/08-consistency-and-caching.md
- Generated: 2026-06-03T07:56:00.066Z
### Source Files
- `CLAUDE.md`
- `internal/tigerfs/fs/metadata_cache.go`
- `internal/tigerfs/fs/path_cache.go`
- `docs/adr/014-query-reduction-caching.md`
- `internal/tigerfs/fs/operations.go`
---
title: "Consistency and caching"
description: "Cross-mount read freshness (no content cache), metadata/stat/path cache TTLs, invalidation on writes, and pipeline cache key isolation."
---
TigerFS treats PostgreSQL as the single source of truth across every mount of the same database: `ReadFile`, export reads, and column reads always query the database, while `Operations` in `internal/tigerfs/fs` caches only metadata (directory entries, stat sizes, catalog lists) with short TTLs and explicit invalidation on writes.
## Design principle
Multiple TigerFS processes can mount one database at the same time. The consistency contract is:
| Data class | Cached? | Cross-mount freshness |
|------------|---------|------------------------|
| Row/column/export **content** | Never | Always current (every read hits PostgreSQL) |
| Directory **listings** (`ReadDir`) | No result cache | Always current |
| **Stat** metadata (size, mode, mtime) | Yes, 2s TTL + write invalidation | Visible on other mounts within ~2s |
| **Path resolution** (parent → row ID) | Yes, 2s TTL + write invalidation | Same as stat cache |
| **Catalog** (schemas, tables, views) | Yes, configurable TTL | Default 10s; explicit invalidate on DDL/build |
| **Structural** (row counts, permissions, PK negatives) | Yes, slower TTL | Default 5m |
| **Column definitions** | Session cache until invalidate | Cleared with `MetadataCache.Invalidate()` |
Never cache row values or export payloads. A write on mount A must be visible to `cat` on mount B immediately. Only metadata may be stale for a bounded window.
## Cache tiers in `fs.Operations`
ADR-014 defines a three-tier query-reduction strategy. The shared `Operations` struct (`internal/tigerfs/fs/operations.go`) implements these plus auxiliary caches:
```mermaid
flowchart TB
subgraph clients [Mount clients]
FUSE[FUSE adapter]
NFS[NFS adapter go-nfs]
end
subgraph ops [fs.Operations]
T1[schemaOnce - Tier 1]
MC[MetadataCache - Tier 2]
SC[statCache - Tier 3]
PC[pathCache]
UC[undoCache]
end
PG[(PostgreSQL)]
FUSE --> ops
NFS --> ops
T1 --> PG
MC --> PG
SC -.->|Stat only| PG
PC -.->|resolve_path| PG
UC -.->|preview only| PG
ops -->|ReadFile ReadDir exports| PG
```
### Tier 1: Schema resolution (`sync.Once`)
`resolveSchema` fills empty `FSContext.Schema` from `current_schema()` once per mount via `schemaOnce`. This removes repeated `SELECT current_schema()` on every RPC (roughly 4–6 queries per operation before caching).
- **Invalidation:** None until mount restart.
- **Scope:** Default-schema paths such as `/users` (not explicit `/.schemas/foo/...` paths).
### Tier 2: `MetadataCache`
`MetadataCache` (`internal/tigerfs/fs/metadata_cache.go`) caches information_schema–style data in two TTL sub-tiers:
| Sub-tier | Config key | Default | Contents |
|----------|------------|---------|----------|
| Catalog | `metadata_refresh_interval` | `10s` | Schemas, table lists, view lists per schema |
| Structural | `structural_metadata_refresh_interval` | `5m` | Row count estimates (`pg_class.reltuples`), batch permissions |
| Per-table (no TTL) | — | Until invalidate | Column lists, primary keys (positive and negative cache) |
`Invalidate()` clears all tiers, PK/column maps, and per-schema lazy caches. Call sites include DDL commit (`write.go`), and synth `.build/` / `.format/` (`build.go`).
Catalog refresh interval. Env: `TIGERFS_METADATA_REFRESH_INTERVAL`. YAML under metadata section in config display.
Structural refresh interval. Env: `TIGERFS_STRUCTURAL_METADATA_REFRESH_INTERVAL`.
### Tier 3: `statCache` (ReadDir-primed stat metadata)
`statCache` stores `Entry` metadata (name, `IsDir`, `Size`, `ModTime`, `Mode`) keyed by `schema + "\x00" + table`, with per-filename entries inside a `tableCache` bucket.
| Behavior | Detail |
|----------|--------|
| TTL | `statCacheTTL` = **2 seconds** (table bucket expires entirely) |
| Prime | `ReadDir` on tables, synth views, and row directories calls `statCache.prime` |
| Lookup | `Stat` checks cache before PostgreSQL (native rows, synth files, pipeline info/export **sizes**) |
| Negative cache | Missing synth/native files can be cached as negatives for up to 2s |
| Row columns | Keys use `schema\x00table/rowPK` prefix; `invalidate` clears table and all `table/` row-level buckets |
`ReadFile` and `readExportFile` **do not** use `statCache` for content—they always query PostgreSQL. Stat may cache the **size** of an export by running `readExportFile` once inside `statExport`, but a subsequent `cat` still fetches fresh data.
### `pathCache` (synth hierarchy)
For parent-pointer synth directories (ADR-017), `pathCache` maps `(parentID, filename) → row UUID` per table, with the same **2s** TTL as stat cache (`pathCacheTTL` in `path_cache.go`). Invalidated alongside `statCache` on directory structure changes (rename, move, delete, undo).
### `undoCache` (preview only)
Undo preview paths (`.undo/.../.info/summary`, affected-file listings) use a **2s** TTL cache over read-only preview queries. **Apply** (`.apply`) re-queries inside `ExecuteUndoTransaction` without using this cache. A narrow stale-preview window is possible if apply commits while a preview `SELECT` is in flight; apply correctness does not depend on cache freshness.
## Cross-mount read freshness
### Content reads (always fresh)
```text
Mount A: echo "new" > /mnt/a/app/note.md
Mount B: cat /mnt/b/app/note.md → queries DB, sees "new" immediately
```
`readFileWithParsed` routes row/column/export/history reads to `db.GetRow`, `db.GetColumn`, pipeline export queries, or synth synthesis—none consult a content cache.
### Directory listings
`ReadDir` always lists from PostgreSQL (or pipeline SQL). It may **prime** `statCache` as a side effect, but the listing itself is not served from cache.
### Stat and `ls -l` staleness
NFS and FUSE issue many `Stat`/`GETATTR` calls per directory entry. Without caching, synth `ls -l` could trigger tens of full-table scans; with ReadDir-primed stat cache, one ReadDir plus cached Stats suffices (ADR-014).
Cross-mount visibility for **metadata**:
- Another mount’s write calls `statCache.invalidate(userSchema, table)` → immediate on that process.
- If invalidation is missed (wrong schema key—see below), negatives can survive up to **2s**.
- Another mount with no write: stat entries expire after **2s** TTL.
macOS mounts use NFS `noac` so the **kernel** does not cache attributes for ~60s. TigerFS still serves GETATTR from its in-memory stat cache (2s). `noac` adds loopback round-trips but does not multiply SQL load.
### FUSE kernel caches (separate layer)
Linux FUSE exposes tunable kernel attribute caching:
| Flag | Default | Role |
|------|---------|------|
| `attr_timeout` | `1s` | FUSE attribute cache |
| `entry_timeout` | `1s` | FUSE dentry cache |
These are independent of TigerFS’s `statCache`. They affect how long the **kernel** may reuse inode attributes, not whether TigerFS queries PostgreSQL for content.
## Write invalidation
Successful writes must drop cached metadata for the affected table so other mounts and subsequent Stats do not serve stale directory structure or false negatives.
### `statCache` and `pathCache`
`WriteFile` documents postconditions: stat/path caches for the table are invalidated (`write.go`). Typical call pattern:
```go
o.statCache.invalidate(fsCtx.Schema, fsCtx.TableName)
o.pathCache.invalidate(fsCtx.Schema, fsCtx.TableName) // synth hierarchy ops
```
Also invalidated on: synth create/update/delete/rename, undo apply, and related synth ops (`synth_ops.go`, `undo.go`).
Pass the **user-facing schema** (the view/workspace schema), not `synth.TigerFSSchema`. Undo and synth paths resolve stats against the user schema; invalidating `tigerfs` leaves stale negatives on the workspace mount for up to 2s. `undo.go` documents this explicitly for apply.
### `MetadataCache`
| Trigger | Action |
|---------|--------|
| DDL `.commit` | `metaCache.Invalidate()` |
| `.build/` / `.format/` synth setup | `invalidateSynthCache()` + `metaCache.Invalidate()` |
| PK DDL | `InvalidatePrimaryKey(schema, table)` |
| TTL expiry | Automatic catalog/structural refresh on next access |
### `undoCache`
Cleared on undo apply (`undoCache.invalidate()`), together with stat/path invalidation under the user schema.
## Pipeline cache key isolation
Pipeline paths build SQL from `FSContext` (filters, order, limits, column projection). **Stat** for pipeline-scoped virtual files must not share cache entries with unfiltered paths on the same table.
TigerFS uses the **full mount path string** as the stat cache filename key for:
- **`.info/` files** — e.g. `.filter/active/true/.info/count` vs `.info/count` (`statInfoFile` / `cachedStat`)
- **`.export/` files** — e.g. `.filter/in_stock/true/.export/json` vs `.export/json` (`statExport`)
Integration tests in `test/integration/command_cases.go` (`ExportAllProductsJSON` → filtered export → `ExportAllProductsJSON_AfterFiltered`) verify that a filtered export stat does not corrupt the unfiltered export size used by NFS `READ`.
`cat` on any export path always runs `readExportFile` against live pipeline state. Stat cache only affects reported **size** for GETATTR; read content is unaffected.
Table-level stat cache keys remain `schema\x00table` for row directories and synth filenames; pipeline state is not part of the table key—only paths that encode pipeline in the **filename key** (info/export) get isolation.
## What stays uncached or lightly cached
| Operation | Behavior |
|-----------|----------|
| `ReadFile` / `cat` | DB query every time |
| `ReadDir` / `ls` | DB (or pipeline) query; may prime stat cache |
| `readExportFile` | Pipeline-aware SQL every time |
| `readColumnFile` | `GetColumn` every time |
| Undo `.apply` | Fresh transaction, no undo preview cache |
| Synth targeted lookups | `RowExistsByColumns` / `GetRowByColumns` on stat miss (ADR-014) |
## Configuration reference
Defaults from `config.Init()` (`internal/tigerfs/config/config.go`):
```yaml
metadata_refresh_interval: 10s
structural_metadata_refresh_interval: 5m
attr_timeout: 1s # FUSE only
entry_timeout: 1s # FUSE only
```
Precedence: defaults → `~/.config/tigerfs/config.yaml` → `TIGERFS_*` env → CLI flags (see [Configuration reference](/configuration-reference)).
## Operational expectations
| Scenario | Expected behavior |
|----------|-------------------|
| Cross-mount `cat` after write | Immediate (no content cache) |
| Cross-mount `ls` after external INSERT | New row may appear up to ~2s late on stat-primed listings; direct `cat` on new path works immediately |
| Cross-mount DDL (new table) | Visible after catalog TTL (default 10s) or `metaCache.Invalidate()` from a TigerFS DDL commit on any mount |
| `ls -l` on synth dir | Fast after one ReadDir; Stats hit stat cache |
| Undo restore, `ls` on macOS NFS | `noac` + TigerFS invalidation; restored rows get fresh `modified_at` for client readdir |
| False `ENOENT` for existing file | Check debug logs for `statCache.isNegative hit`; suspect stale negative or wrong-schema invalidation |
## Debugging
Enable debug logging on the mount (`--debug`) and look for:
- `statCache.lookup hit` / `statCache.isNegative hit`
- `statCache.invalidate` with `schema` and `table`
- `Metadata cache catalog expired, refreshing`
Stress runs may log NFS monotonicity warnings on `.log/.last/N/.export/json`; those reflect macOS NFS client staleness, not TigerFS content caching (see project stress-test notes in `CLAUDE.md`).
## Related pages
Pipeline path grammar and how FSContext drives SQL for exports and listings.
FUSE vs NFS adapters, noac, and NFS write/stat interaction with stat cache.
Synth ReadDir priming, pathCache, and targeted stat queries.
Undo preview cache, apply path freshness, and post-undo invalidation.
Stale mounts, debug logging, and metadata refresh recovery.
Full config keys including metadata and FUSE timeout fields.
---
## 09. Platform backends
> Linux FUSE vs macOS in-process NFS (go-nfs), write/commit model on NFS, mount registry, and adapter delegation to fs.Operations.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/09-platform-backends.md
- Generated: 2026-06-03T07:55:43.504Z
### Source Files
- `internal/tigerfs/fuse/adapter.go`
- `internal/tigerfs/nfs/handler.go`
- `internal/tigerfs/nfs/mount_darwin.go`
- `internal/tigerfs/mount/registry.go`
- `docs/spec.md`
- `docs/adr/001-fuse-library.md`
---
title: "Platform backends"
description: "Linux FUSE vs macOS in-process NFS (go-nfs), write/commit model on NFS, mount registry, and adapter delegation to fs.Operations."
---
TigerFS selects a VFS backend at compile time: Linux mounts through `hanwen/go-fuse/v2` (default `FUSE+Operations` via `fuse.MountOps`); macOS runs an in-process NFS v3 server (`willscott/go-nfs`) and attaches it with `mount_nfs` to `127.0.0.1`. Both paths construct the same `fs.Operations` instance and route path semantics, SQL, and error codes through that core—adapters only translate POSIX/FUSE or NFS/billy calls.
## Platform matrix
| OS | Backend | Library | Default mount entry | CLI flag |
| --- | --- | --- | --- | --- |
| Linux | FUSE | `github.com/hanwen/go-fuse/v2` | `fuse.MountOps` → `OpsFS` + `FSAdapter` | `--legacy-fuse` selects `fuse.Mount` (specialized node tree) |
| macOS | In-process NFS v3 | `github.com/willscott/go-nfs` | `nfs.Mount` → `Server` + `OpsFilesystem` | NFS-only on Darwin (`mount_linux.go` stubs NFS with an error) |
| Windows (spec) | WinFsp | Documented in spec; not the focus of current `cmd` build tags | — | — |
`mountFilesystem` is implemented in platform-specific `cmd/mount_*.go` files. `tigerfs version` reports the active backend (`Backend: NFS` on Darwin, FUSE on Linux).
ADR-001 records why `bazil.org/fuse` was rejected: it failed to compile on macOS. Production FUSE uses `hanwen/go-fuse/v2` for explicit `syscall.Errno` mapping and cross-platform builds.
## Architecture: shared core, thin adapters
```mermaid
flowchart TB
subgraph clients["OS clients"]
LinuxTools["Linux: VFS → FUSE"]
MacTools["macOS: VFS → NFS client"]
end
subgraph adapters["Platform adapters"]
FUSEAdapter["fuse.FSAdapter / OpsNode"]
NFSServer["nfs.Server"]
BillyFS["nfs.OpsFilesystem"]
StableH["nfs.StableHandler"]
end
subgraph core["Shared filesystem logic"]
Ops["fs.Operations"]
DB["db.Client → PostgreSQL"]
end
LinuxTools --> FUSEAdapter
MacTools --> NFSServer
NFSServer --> StableH --> BillyFS
FUSEAdapter --> Ops
BillyFS --> Ops
Ops --> DB
```
New filesystem behavior belongs in `internal/tigerfs/fs/`, not in adapter packages. The FUSE tree still contains a legacy specialized node hierarchy (`TableNode`, `PipelineNode`, `StagingNode`, etc.) used only with `--legacy-fuse`; the default Linux path uses a single `OpsNode` type that mirrors NFS’s one-adapter design.
## Linux: FUSE + Operations (default)
`fuse.MountOps` wires the stack:
1. `db.NewClient` connects to PostgreSQL.
2. `fs.NewOperations(cfg, dbClient)` with `SetMountPoint` / `SetUserID`.
3. `NewFSAdapter(ops)` converts `fs.Entry` / `fs.FSError` to FUSE types.
4. `newOpsNode(adapter)` is the root; child paths are derived from the inode tree (`currentPath()`), not cached strings, so renames stay consistent after `MvChild`.
5. `fs.Mount` with `AttrTimeout` / `EntryTimeout` from config (FUSE-only kernel caches).
`FSAdapter` implements `ReadDir`, `Stat`, `ReadFile`, `WriteFile`, `Delete`, `Mkdir`, and `Rename` by delegating to `fs.Operations`. `ErrorToErrno` maps `fs.ErrorCode` to POSIX errno and logs hints via zap (FUSE cannot return messages to callers).
### FUSE_INTERRUPT handling
Read and write entry points call `decoupleFromRequestCancel(ctx)` so advisory `FUSE_INTERRUPT` signals (for example from Go’s `SIGURG` under GC) do not cancel in-flight `pgx` queries and surface spurious `EIO`. Statement timeouts and pool close on unmount still bound work.
### Legacy FUSE (`--legacy-fuse`)
`fuse.Mount` builds `RootNode` and specialized nodes with direct DB paths for tables, pipelines, staging, and exports. Use only when debugging parity with the old tree; default mounts should stay on `MountOps`.
## macOS: in-process NFS v3
`nfs.Mount` (Darwin build tag):
1. Connects DB and creates `nfs.NewServer`.
2. `Server.Start` listens on `127.0.0.1:0` (ephemeral port, localhost only).
3. Serves RPCs with `nfs.Serve(listener, NewStableHandler(billyFS))`.
4. Runs `mount_nfs` against `127.0.0.1:/` with tuned options.
### `mount_nfs` options (behavioral)
| Option | Role |
| --- | --- |
| `vers=3`, `tcp` | NFS v3 over TCP to loopback server |
| `port` / `mountport` | Dynamic port from in-process server |
| `wsize=131072`, `rsize=131072` | 128KB chunks—fewer RPCs than macOS defaults; avoids 1MB+ same-process GC issues |
| `noac` | Disables client attribute cache so cross-mount/undo changes are visible; GETATTR still hits TigerFS stat cache (~2s TTL) |
| `soft`, `timeo=300`, `retrans=2` | Timeouts instead of indefinite hang on slow commits |
| `nolocks`, `locallocks` | go-nfs does not implement NLM |
| `noresvport` | Non-root mounts |
### `StableHandler` and READDIR consistency
`StableHandler` embeds `NullAuthHandler` and overrides handle encoding: paths fit in the 64-byte NFS v3 handle when possible (version 1, 4-byte aligned for macOS); longer paths use SHA-256 fallback with a small in-memory map.
A 5-second `readdirCache` keyed by path + verifier prevents `BadCookie` when go-nfs paginates READDIRPLUS with conservative byte estimates—critical for non-deterministic listings such as `.sample/` (`ORDER BY RANDOM()`).
### `OpsFilesystem` (billy → Operations)
`NewOpsFilesystem` wraps the same `fs.Operations` as FUSE. Directory ops (`ReadDir`, `Stat`, `Mkdir`, `Rename`, `Delete`) call through immediately. File writes use a persistent `cachedFile` map (ADR-010) because go-nfs has no real open state.
## NFS write and commit model
NFS v3 is stateless. go-nfs synthesizes **`Open → Write → Close` per WRITE RPC** because `billy.Filesystem` requires it. TigerFS commits on **`memFile.Close`** when the fabricated handle’s refCount reaches zero (and the buffer is dirty).
```text
NFS WRITE RPC #1 WRITE RPC #2 ...
│ │
▼ ▼
OpenFile (cache) OpenFile (ref++)
Write → buffer grow Write → buffer grow
Close → commit full Close → commit full buffer again
buffer to DB (O(n²) bytes written for n chunks)
```
| Topic | Behavior |
| --- | --- |
| Durability | Each Close with dirty data calls `ops.WriteFile`—data is in PostgreSQL after every RPC, not deferred to a final COMMIT |
| Stat during write | Dirty/truncated cache entries return in-flight size from memory (required for READ after WRITE before Close) |
| Empty Close | Skipped for truncate-before-write (SETATTR→WRITE) except DDL `sql` files |
| DDL triggers | `.test` / `.commit` / `.abort` fire on Close; `Chtimes` (touch) also fires but dedupes via `triggered` flag |
| Large sequential writes | Above `nfs_streaming_threshold` (default 10MB), buffer commits and clears mid-write |
| Random writes | Capped by `nfs_max_random_write_size` (default 100MB) |
| Crashed clients | Reaper every `nfs_cache_reaper_interval` (default 30s) force-commits idle entries after `nfs_cache_idle_timeout` (default 5m) |
| Mitigation | 128KB `wsize`/`rsize`; files under one chunk avoid O(n²) amplification |
Stress runs and heavy writes may log NFS monotonicity warnings (stale `.log/.last/N/.export/json` snapshots). These are treated as expected macOS NFS client behavior, not TigerFS data bugs. See the troubleshooting page.
Long-term direction in spec: patch go-nfs to return `UNSTABLE` on WRITE and flush on NFS COMMIT for O(n) semantics.
## FUSE vs NFS write path comparison
| Aspect | FUSE (`OpsFileHandle`) | NFS (`memFile` + cache) |
| --- | --- | --- |
| Buffer | Per-handle memory; flush on `Flush` | Shared `cachedFile` per path |
| Commit trigger | FUSE flush/close semantics | Every go-nfs Close (per WRITE RPC) |
| Read while writing | Standard FUSE open state | Read-only opens bypass cache and hit DB for formatted output |
| Create validation | `ValidateCreate` before handle | Same—rejects bare-path row creates that would confuse macOS dentry types |
## Mount registry
Long-lived `tigerfs mount` processes register themselves so `unmount`, `status`, `list`, `info`, and `fork` can find active mounts without scanning `/proc`.
| Field | Purpose |
| --- | --- |
| `mountpoint` | Absolute mount path |
| `pid` | Serving process; `ListActive` / `Cleanup` use signal 0 liveness |
| `database` | Sanitized connection string (no password) |
| `service_id`, `cli_backend` | Cloud backend metadata for `tigerfs info` / `fork` |
| `auto_created` | If true, mountpoint dir removed after unmount |
| `start_time` | Uptime display |
Persistence path (`mount.DefaultRegistryPath`):
- Linux: `~/.config/tigerfs/mounts.json`
- macOS: `~/Library/Application Support/tigerfs/mounts.json`
File mode `0600`; parent dir `0700`. `Register` replaces duplicate mountpoints; `Unregister` on graceful shutdown; stale PIDs removed by `Cleanup` or filtered in `ListActive`.
## Configuration surfaces
| Key / flag | Applies to |
| --- | --- |
| `attr_timeout`, `entry_timeout` / `--attr-timeout`, `--entry-timeout` | FUSE kernel caches only |
| `nfs_streaming_threshold`, `nfs_max_random_write_size`, `nfs_cache_reaper_interval`, `nfs_cache_idle_timeout` | NFS `OpsFilesystem` cache |
| `--legacy-fuse` | Linux only; legacy node tree |
macOS does not use FUSE timeout flags; attribute freshness relies on `noac` plus TigerFS metadata caches (see consistency page).
## Verification
```bash
tigerfs version
```
Expect `Backend: NFS` on macOS and FUSE on Linux.
```bash
tigerfs list
tigerfs status /path/to/mountpoint
```
Active entries require a live PID in `mounts.json`.
```bash
tigerfs mount postgres://localhost/db /mnt/test
# vs
tigerfs mount postgres://localhost/db /mnt/test --legacy-fuse
```
Default logs `Using FUSE+Operations backend for Linux`.
Integration tests auto-select `MountMethodFUSE` on Linux and `MountMethodNFS` on macOS (`test/integration/setup.go`); do not prefix tests with `TestNFS_` unless exercising NFS-only behavior.
## Related pages
Platform prerequisites: Linux FUSE group membership vs macOS NFS (no extra install).
How `fs.Operations` maps mount paths to SQL—shared by both backends.
Cross-mount freshness, stat/path cache TTLs, and why NFS uses `noac`.
`mount`, `unmount`, `list`, `status`, and `--legacy-fuse`.
FUSE timeouts and NFS cache tuning keys.
Stale mounts, force unmount, NFS monotonicity warnings, `--debug`.
Building, `go test`, and integration mount method detection.
---
## 10. File-first workspaces
> Create workspaces, read/write markdown and plaintext, directories as state, frontmatter semantics, and backing-table access via .tables/.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/10-file-first-workspaces.md
- Generated: 2026-06-03T07:56:46.406Z
### Source Files
- `docs/file-first.md`
- `internal/tigerfs/fs/synth/markdown.go`
- `internal/tigerfs/fs/synth/plaintext.go`
- `internal/tigerfs/fs/build.go`
- `internal/tigerfs/fs/write.go`
- `test/integration/fs_operations_test.go`
---
title: "File-first workspaces"
description: "Create workspaces, read/write markdown and plaintext, directories as state, frontmatter semantics, and backing-table access via .tables/."
---
TigerFS file-first workspaces are synthesized PostgreSQL views (`internal/tigerfs/fs/synth/`) that expose each row as a file under `//`. Creating a workspace writes a format string to `/.build/`; TigerFS provisions a backing table in the `tigerfs` schema, a user-schema view, and optional history infrastructure. File content is always read from the database on each `ReadFile`; metadata may be cached with invalidation on writes.
## Workspace layout
A `.build/` workspace creates two mount surfaces for the same data:
| Path | Role |
|------|------|
| `//` | File-first: markdown or plaintext files, optional subdirectories |
| `/.tables//` | Data-first: row directories, column files, pipeline capabilities |
```text
mount/
├── .build/ write-only (listing empty)
├── .tables/ tigerfs-schema backing tables
│ └── notes/ data-first row/column access
└── notes/ file-first workspace (view)
├── hello.md
├── tutorials/
│ └── intro.md
├── .history/ (history-enabled only)
├── .log/
├── .savepoint/
└── .undo/
```
History-enabled workspaces add `.history/`, `.log/`, `.savepoint/`, and `.undo/` at the **workspace root only**; subdirectories list real files and child directories, not those control dirs.
Pipeline capabilities (`.filter/`, `.export/`, `.by/`, and similar) are **blocked** on file-first workspace paths and return `ErrInvalidPath` with a hint to use `/.tables//...`. Workspace control surfaces that remain allowed include `.info/`, `.history/`, `.log/`, `.savepoint/`, `.undo/`, and `.format/` (see `rejectDataFirstCapOnSynthWorkspace` in `internal/tigerfs/fs/operations.go`).
## Create a workspace
`/.build/` is mount-root only; it is rejected under `/.schemas//.build/` (ADR-015).
Write a format (and optional `history`) to a virtual file `/.build/`:
```bash
echo "markdown,history" > /mnt/db/.build/notes
echo "markdown" > /mnt/db/.build/posts
echo "plaintext" > /mnt/db/.build/snippets
echo "history" > /mnt/db/.build/notes # add history to existing app
```
```bash
ls /mnt/db/ # expect and .tables
ls /mnt/db/notes/ # workspace directory (empty until files exist)
ls /mnt/db/.tables/ # expect backing table name
```
`writeBuildFile` (`internal/tigerfs/fs/build.go`) parses features via `synth.ParseFeatureString`, requires TimescaleDB when `history` is requested, executes generated DDL in `tigerfs`, and invalidates synth/metadata caches.
### Supported `.build/` inputs
| Written content | Effect |
|-----------------|--------|
| `markdown` or `md` | Markdown workspace: `tigerfs.` table + `` view, comment `tigerfs:md` |
| `txt`, `text`, `plaintext`, `plain` | Plaintext workspace, comment `tigerfs:txt` |
| `markdown,history` | Markdown + versioned history tables |
| `history` alone | Add history to an existing app (table must already exist in `tigerfs`) |
| `tasks`, `native` | Rejected — unsupported for `.build/` |
Plaintext and markdown `.build/` tables include `parent_id`, `filetype` (`file` \| `directory`), `filename`, `body`, `encoding`, and timestamps. Markdown tables also include `title`, `author`, and `headers JSONB` for extra frontmatter keys (`internal/tigerfs/fs/synth/build.go`).
`history` requires the TimescaleDB extension. Without it, `.build/` writes fail with a structured error and install hint.
## Add file-first to an existing table
For tables that already exist in the user schema, write to `//.format/` instead of `.build/`:
```bash
echo ok > /mnt/db/posts/.format/markdown
# Creates view posts_md (table + "_md" suffix)
```
Supported `.format/` names: `markdown`, `txt` (`synth.AvailableFormats()`). The view selects all columns from the existing table; column roles are auto-detected from names (see below). This path does **not** create a new `tigerfs` backing table.
## Markdown and plaintext files
### Markdown
Each file is YAML frontmatter (from columns) plus a body column:
```markdown
---
title: Getting Started
author: alice
tags:
- tutorial
draft: false
---
# Getting Started
Body text stored in the `body` column.
```
Synthesis: `synth.SynthesizeMarkdown` (`internal/tigerfs/fs/synth/markdown.go`). Parsing: `synth.ParseMarkdown` + `synth.MapToColumns`.
### Plaintext
No frontmatter — the entire file maps to the body column (`synth.SynthesizePlainText`, `synth.ParsePlainText` in `internal/tigerfs/fs/synth/plaintext.go`).
### Binary content
If file bytes contain NUL or invalid UTF-8, `writeSynthFile` stores base64 in `body` and sets `encoding` to `base64` when that column exists.
## Frontmatter semantics
Column roles are detected by naming convention (first match wins) in `synth.DetectColumnRoles` (`internal/tigerfs/fs/synth/columns.go`):
| Role | Convention (priority order) | Required |
|------|----------------------------|----------|
| Filename | `filename`, `name`, `title`, `slug` | Yes |
| Body | `body`, `content`, `description`, `text` | Yes |
| Modified / created | `modified_at`/`updated_at`, `created_at` | No |
| Extra headers | `headers` (JSONB) | No |
| Hierarchy | `filetype`, `parent_id` | On `.build/` tables |
Remaining markdown columns (except PK, timestamps, `filetype`, `parent_id`, `encoding`) become frontmatter fields in schema order.
### Write behavior
| Field class | On edit, if key omitted from frontmatter |
|-------------|------------------------------------------|
| Known frontmatter columns (`title`, `author`, …) | **Unchanged** — `UpdateRow` only sets columns present in the parsed map |
| `headers` JSONB keys | **Removed** — full replace; omitting a key drops it |
| `body` | Always replaced with file body |
| Timestamps | Not in frontmatter; drive `ls -l` mtimes only |
To clear a known column, set it explicitly (e.g. `title: ""`). Unknown frontmatter keys without a `headers` column cause a parse error.
Discover columns on the backing table:
```bash
cat /mnt/db/.tables/notes/.info/columns
```
## Read and write files
Standard tools work on the workspace path:
```bash
ls /mnt/db/notes/
cat /mnt/db/notes/hello-world.md
grep -r "TODO" /mnt/db/notes/
cat > /mnt/db/notes/new-post.md << 'EOF'
---
title: New Post
author: bob
---
Content here.
EOF
mv /mnt/db/notes/old.md /mnt/db/notes/new.md
rm /mnt/db/notes/unwanted.md
```
`ReadFile` always queries the database (no content cache). Writes invalidate stat and path caches for the view schema/table.
### Reserved filenames
Leaf names matching TigerFS capability directories (`.history`, `.log`, `.savepoint`, `.undo`, `.info`, `.by`, `.filter`, `.export`, …) are rejected with `ErrPermission`. `ReadDir` filters rows that collide with reserved names so virtual control dirs take precedence.
## Directories as state
`.build/` markdown and plaintext apps use the relational directory model (ADR-017): each directory is a row with `filetype='directory'`; files reference `parent_id`. Path segments under the workspace map to `parent_id` + leaf `filename`.
```bash
mkdir /mnt/db/notes/tutorials
# Or deep paths via mkdir -p semantics:
```
| Operation | Behavior |
|-----------|----------|
| `mkdir` | Inserts `filetype=directory` row; does not auto-create missing ancestors (explicit per segment, like POSIX) |
| `WriteFile` via FUSE/NFS | Parent must exist before create (kernel enforces `O_CREAT` parent) |
| `WriteFileEnsureDirs` / `MkdirAll` | API helpers that create missing ancestors (tests, batch import) |
| `mv` directory | Atomic rename: updates directory row and descendant paths (`TestSynth_HierarchicalRenameDir`) |
| `rmdir` | Fails with not-empty if children exist |
Integration coverage: `test/integration/synthesized_test.go` (`TestSynth_HierarchicalMkdir`, `TestSynth_HierarchicalWriteFile`, `TestSynth_HierarchicalRenameDir`, and related tests).
```mermaid
flowchart LR
subgraph mount["Mount path"]
W["/notes/a/b/doc.md"]
end
subgraph fs["fs.Operations"]
P["parseSynthContent"]
WSF["writeSynthFile"]
end
subgraph db["PostgreSQL tigerfs.notes"]
D["directory rows parent_id"]
F["file rows body + frontmatter cols"]
end
W --> WSF --> P
WSF --> D
WSF --> F
```
## Backing table access via `.tables/`
`/.tables//` rewrites paths to schema `tigerfs` with the same logical table name as the workspace. Use it for data-first exploration while keeping agents on file paths:
```bash
ls /mnt/db/notes/ # file-first
ls /mnt/db/.tables/notes/ # row IDs + capabilities
cat /mnt/db/.tables/notes//title
```
`TestSynth_TablesRoute` in `test/integration/synthesized_test.go` verifies round-trip between `/posts/hello-world.md` and `/.tables/posts//title`.
For `.format/` views (e.g. `posts_md`), the original table stays data-first at `/posts/`; the synthesized view is file-first at `/posts_md/`.
## Optional history
Append `,history` at `.build/` time (or write `history` later) to enable `.history/`, `.log/`, `.savepoint/`, and `.undo/` at the workspace root. Requires TimescaleDB.
Version browsing, savepoint JSON, undo preview/apply, and per-user filters.
## Implementation map
| Concern | Package / entry |
|---------|-----------------|
| Build / format CLI surface | `internal/tigerfs/fs/build.go` |
| Synth read/write/rename | `internal/tigerfs/fs/synth_ops.go` |
| Path parsing `/.build/`, `/.tables/` | `internal/tigerfs/fs/path.go` |
| DDL generation | `internal/tigerfs/fs/synth/build.go` |
| Format detection | `internal/tigerfs/fs/synth/format.go` |
Human-oriented examples and recipes also appear in `docs/file-first.md` and `skills/tigerfs/files.md`.
## Related pages
Mode detection, `.build/` vs `.format/`, and when a path is a workspace vs a table.
View comments, backing schema, and column/frontmatter mapping details.
Full mount hierarchy and how `fs.Operations` maps paths to SQL.
Why `cat` always hits the DB and which metadata caches apply to workspaces.
Kanban, knowledge base, and other file-first patterns.
---
## 11. Data-first exploration
> Row/column reads, index navigation (.by), pagination (.first/.last/.sample), PATCH writes, import/export, and large-table safety limits.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/11-data-first-exploration.md
- Generated: 2026-06-03T07:57:32.192Z
### Source Files
- `docs/data-first.md`
- `internal/tigerfs/db/query.go`
- `internal/tigerfs/db/pipeline.go`
- `internal/tigerfs/fs/operations.go`
- `internal/tigerfs/format/convert.go`
- `test/integration/crud_test.go`
---
title: "Data-first exploration"
description: "Row/column reads, index navigation (.by), pagination (.first/.last/.sample), PATCH writes, import/export, and large-table safety limits."
---
Data-first mode maps each PostgreSQL table under the mount to a directory tree: primary keys become path segments, row bodies are readable as `.json`/`.csv`/`.tsv`/`.yaml` files or as per-column files, and dot-capability directories (`.by`, `.filter`, `.first`, `.export`, `.import`) compile into SQL in `internal/tigerfs/db/pipeline.go` and `internal/tigerfs/fs/operations.go`. Reads always query PostgreSQL; metadata caches do not store row content.
```text
/mnt///
├── # Row directory (or .json / .tsv / …)
│ ├── # Numeric types: no extension
│ ├── .txt # text/varchar
│ ├── .json # json/jsonb
│ └── .bin # bytea
├── .by/ # Indexed column navigation
├── .filter/ # Any-column equality filters
├── .first/ .last/ .sample/
├── .columns/ .order/ .export/
├── .import/ # Bulk load (not pipeline-composable)
└── .info/ # ddl, schema, columns, count
```
File-first workspaces (markdown/plaintext under `.build/`) reject workspace-level `.by`, `.filter`, and `.export` paths. Use `/.tables//` for data-first access to the backing table without bypassing history semantics on normal workspace files.
## Prerequisites
- A mounted TigerFS database (see [Quickstart](/quickstart)).
- Default schema tables visible under the mount root (or an explicit schema path).
- For large tables: configure `dir_listing_limit`, `dir_filter_limit`, and `query_timeout` before relying on `ls` over millions of rows.
## Reading rows and columns
| Path pattern | Operation | Backend |
|--------------|-----------|---------|
| `//.json` | Full row, JSON | `db.GetRow` + `format.RowToJSON` |
| `//.tsv` / `.csv` / `.yaml` | Full row, tabular/YAML | `format.RowTo*` |
| `///` or `.txt` | Single column | `db.GetColumn` + `format.ConvertValueToText` |
| `//.info/count` | Row count | Metadata query |
| `//.info/columns` | Column names (one per line) | Column cache |
```bash
ls /mnt/db/users/ # PKs (capped by dir_listing_limit)
cat /mnt/db/users/42.json # Entire row
cat /mnt/db/users/42/email.txt # One column
cat /mnt/db/users/.info/count # Total rows
```
Row reads go through `Operations.ReadFile` → `readRowFile` / `readColumnFile`, which call the database on every read. Column files append a trailing newline on output; empty writes to a column set NULL (`TestCRUDNullHandling`, `TestFSOperations_WriteFile_NullColumn`).
### Row-as-directory dot formats
Inside `///`, dot-prefixed files expose the full row in alternate formats:
```bash
cat /mnt/db/users/42/.json
cat /mnt/db/users/42/.csv
```
## Index navigation with `.by/`
`.by/` exposes only columns that have indexes. Listing is index-backed and uses `dir_listing_limit` for distinct-value queries (same cap as table row listings in the shared operations path).
```bash
ls /mnt/db/orders/.by/ # Indexed columns
ls /mnt/db/orders/.by/status/ # Distinct indexed values
ls /mnt/db/orders/.by/status/pending/ # Rows matching status=pending
cat /mnt/db/orders/.by/customer_id/7.json # Row via index path
```
Each `.by///` segment adds an **indexed** filter (`FilterCondition.Indexed = true` in `db.QueryParams`). Filters AND-combine with prior `.by/` and `.filter/` segments.
### Composite indexes
Two equivalent styles when a multi-column index exists:
```bash
# Sequential (two filters)
.by/last_name/Smith/.by/first_name/John/
# Composite (single segment, matches index column order)
.by/last_name.first_name/Smith.John/
```
Composite syntax is preferred when it matches an existing composite index definition.
## Pagination: `.first/`, `.last/`, `.sample/`
Pagination capabilities appear as numeric subdirectories. They apply at the current pipeline context and push down to SQL (`LimitFirst`, `LimitLast`, `LimitSample` in `db/pipeline.go`).
| Capability | Default ordering | SQL effect |
|------------|------------------|------------|
| `.first/N/` | Primary key ascending | `ORDER BY pk ASC LIMIT N` |
| `.last/N/` | Primary key descending | `ORDER BY pk DESC LIMIT N` |
| `.sample/N/` | Random subset | `ORDER BY RANDOM() LIMIT N` |
```bash
ls /mnt/db/events/.first/100/ # First 100 PKs
ls /mnt/db/events/.last/50/ # Last 50 PKs
ls /mnt/db/events/.sample/25/ # Random 25 PKs
```
### Nested pagination
Chaining limits builds subqueries (`NeedsSubquery()` / `buildNestedPipelineSQL`):
| Path | Meaning |
|------|---------|
| `.first/100/.last/50/` | Last 50 of the first 100 rows (rows 51–100 by PK order) |
| `.last/100/.first/50/` | First 50 of the last 100 rows |
| `.first/1000/.sample/50/` | Random 50 drawn from the first 1000 |
Post-limit filters are allowed (e.g. `.first/100/.filter/status/active/` filters within the capped set).
### Ordering interaction
After `.order//`, only `.first/`, `.last/`, `.sample/`, `.columns/`, and `.export/` may follow. Use `.first/` for ascending sort results and `.last/` for descending on the ordered column.
## Pipeline queries and export
Capability segments between the table root and a row listing or export file form one pipeline. `FSContext` in `internal/tigerfs/fs/context.go` tracks filters, order, limits, column projection, and terminal state; `ToQueryParams()` feeds `db.QueryRowsPipeline` / `QueryRowsWithDataPipeline`.
```bash
cat /mnt/db/orders/.by/customer_id/42/.by/status/pending/.order/created_at/.last/10/.export/json
```
Rough SQL shape:
```sql
SELECT * FROM orders
WHERE customer_id = $1 AND status = $2
ORDER BY created_at DESC
LIMIT 10
```
### Terminal and redundant chains
Rules enforced at parse time (`path.go`, `FSContext.CanAdd*`):
| Rule | Example |
|------|---------|
| `.export/` is terminal | Nothing after `.export/json` |
| After `.columns/`, only `.export/` | `.columns/id,name/.filter/...` invalid |
| No double `.first/` or `.last/` | Use a single smaller limit |
| No limit after `.sample/` | Use `.sample/N/` with smaller N |
| No second `.order/` | Second replaces first; use one `.order/` |
| `.import/` not in pipelines | Bulk import is a separate write path |
Export formats under `.export/`: `csv`, `tsv`, `json`, `yaml`, plus `.with-headers/` variants for csv/tsv. Reading an export path runs the full pipeline and serializes all matching rows.
Deep pipeline grammar, `PathType` resolution, and ADR-level SQL pushdown are documented on [Capability directories](/capability-directories).
## `.by/` vs `.filter/`
| Aspect | `.by/` | `.filter/` |
|--------|--------|------------|
| Columns listed | Indexed columns only | All table columns |
| Value listing | Index scan, `dir_listing_limit` cap | `DISTINCT` query, `dir_filter_limit` cap |
| Performance | Predictable index use | May scan large tables |
| Typical use | Known indexed lookups | Ad-hoc equality on any column |
```bash
ls /mnt/db/users/.by/email/ # Fast value listing
ls /mnt/db/users/.filter/notes/ # Any column; may hit safety limits
```
For `.filter//` on very large tables, when the cached row estimate exceeds `dir_filter_limit` (default 100,000), legacy FUSE mounts surface a `.table-too-large` indicator file instead of enumerating distinct values. Direct paths still work:
```bash
cat /mnt/db/large_events/.filter/type/click/.first/100/.export/json
```
## Writing data (PATCH semantics)
Structured row formats use **PATCH** semantics: only columns present in the payload are updated; omitted columns are unchanged (`format.ParseJSON`, `ParseCSVWithHeader`, `ParseTSVWithHeader`; `Operations.writeRowFile` → `db.UpdateRow`).
| Write target | Semantics | Notes |
|--------------|-----------|-------|
| `/.json` / `.yaml` / `.csv` / `.tsv` | PATCH by column keys / header row | Insert if row missing; format suffix required for new rows |
| `//` or `.txt` | Single-column replace | Empty content → NULL |
| `/` (no extension), existing row | PUT in schema column order | Bare path; TSV values in column order |
| `/` (no extension), new row | Rejected | Use `/.tsv` (or `.json`, etc.) to avoid NFS inode conflicts |
```bash
echo 'new@example.com' > /mnt/db/users/42/email.txt
echo '{"name":"Ada"}' > /mnt/db/users/42.json
echo -e 'email\tname\na@b.com\tAda' > /mnt/db/users/42.tsv
mkdir /mnt/db/users/99 && echo 'x@y.com' > /mnt/db/users/99/email.txt
rm -r /mnt/db/users/99/
```
Writes invalidate stat/path caches for the table schema (`statCache.invalidate`); cross-mount reads still see fresh row data from PostgreSQL.
Text primary keys ending in `.json`, `.csv`, `.tsv`, or `.yaml` are ambiguous with format extensions. Prefer the row-directory view or escape-aware tooling.
## Bulk import and export
Import is **write-only** under `.import/` and is not composable with pipeline segments.
| Mode path | Behavior |
|-----------|----------|
| `.import/.sync/.csv` | Upsert: update existing PKs, insert new |
| `.import/.overwrite/.csv` | Truncate table, then load |
| `.import/.append/.csv` | Insert additional rows only |
Formats: `csv`, `tsv`, `json`, `yaml` under each mode; `.no-headers/` subpaths accept csv/tsv without a header row (column order from table schema).
```bash
cat > /mnt/db/staging/.import/.overwrite/users.csv <<'EOF'
id,name,email
1,Alice,alice@example.com
2,Bob,bob@example.com
EOF
```
Export reads use `.export/` at the end of a pipeline (or on a bare table with implicit limits). Verified modes in integration tests: sync, append, overwrite (`TestFSOperations_Import_*`).
## Large-table safety limits
Configuration lives in `~/.config/tigerfs/config.yaml` and `TIGERFS_*` environment variables (`internal/tigerfs/config/config.go`).
| Key | Default | Effect |
|-----|---------|--------|
| `dir_listing_limit` | `1000` | Max PKs returned in `ls` on a table; max distinct values listed under `.by//` |
| `dir_filter_limit` | `100000` | Max distinct values fetched for `.filter//`; tables above estimate skip value listing (`.table-too-large` on legacy FUSE) |
| `dir_writing_limit` | `100000` | Bulk write safety threshold |
| `query_timeout` | `30s` | Statement timeout for queries |
| `max_pipeline_depth` | `10` | After this depth, pipeline capability dirs are hidden (rows still list) to stop runaway `find`/agent recursion |
| `no_filename_extensions` | `false` | Disable `.txt`/`.json`/`.bin` column suffixes |
```yaml
filesystem:
dir_listing_limit: 1000
dir_filter_limit: 100000
query_timeout: 30s
```
```bash
export TIGERFS_DIR_LISTING_LIMIT=5000
export TIGERFS_DIR_FILTER_LIMIT=100000
export TIGERFS_QUERY_TIMEOUT=60s
```
Use pagination instead of bare `ls`:
```bash
ls /mnt/db/events/.first/100/
cat /mnt/db/events/.first/100/.export/json
```
When `ls .filter/col/` shows `.table-too-large`, jump directly to the value:
```bash
cat /mnt/db/events/.filter/type/click/.last/50/.export/csv
```
```bash
cat /mnt/db/events/.info/count
```
Legacy Linux FUSE (`legacy_fuse: true`) additionally refuses full table `ls` when `pg_class` row estimate exceeds `dir_listing_limit` (logged as "Table too large for directory listing"). The default shared `fs.Operations` path (including macOS NFS) returns up to `dir_listing_limit` PKs plus capability directories.
## File-first backing tables via `.tables/`
Synthesized workspaces store rows in the `tigerfs` schema. Data-first operations on that data use:
```bash
ls /mnt/db/.tables/notes/
cat /mnt/db/.tables/notes/.info/columns
```
Writes under `.tables//` bypass workspace history triggers. They do not appear in `.log/`, `.history/`, or undo paths. Use the file-first workspace for normal edits; reserve `.tables/` for diagnostics and repairs.
## Verification signals
| Action | Expected signal |
|--------|-----------------|
| `cat …/.json` after edit | Updated keys only; other fields unchanged (PATCH) |
| `cat …/.info/count` after import overwrite | Count matches imported rows |
| `ls …/.by//` | Only indexed columns appear |
| Pipeline export | Single JSON array / CSV with header, no trailing capabilities |
| Blocked workspace pipeline | Error with hint mentioning `.tables/` |
Integration coverage: `test/integration/crud_test.go` (insert/select/update/delete, partial updates), `test/integration/pipeline_test.go` (`.by`, nested pagination, `.sample`), `test/integration/fs_operations_test.go` (import modes, PATCH via pipeline column write), `test/integration/synthesized_test.go` (workspace vs `.tables/` gates).
## Related pages
Pipeline grammar, chaining matrix, SQL pushdown, and PathType resolution.
TSV/CSV/JSON/YAML encoding, PATCH details, NULL rules, and type extensions.
When mounts are workspaces vs raw tables and how mode detection works.
Why reads are always fresh and which metadata caches invalidate on writes.
Full config struct, env vars, and precedence.
---
## 12. History, savepoints, and undo
> Enable history on workspaces, .history/.log paths, savepoint JSON, .undo preview/apply, per-user filters, auto-savepoints, and migration boundaries.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/12-history-savepoints-and-undo.md
- Generated: 2026-06-03T07:57:05.739Z
### Source Files
- `docs/history.md`
- `internal/tigerfs/fs/undo.go`
- `internal/tigerfs/fs/history.go`
- `internal/tigerfs/fs/auto_savepoint_test.go`
- `test/integration/undo_test.go`
- `docs/adr/016-undo-and-recovery.md`
---
title: "History, savepoints, and undo"
description: "Enable history on workspaces, .history/.log paths, savepoint JSON, .undo preview/apply, per-user filters, auto-savepoints, and migration boundaries."
---
History-enabled file-first workspaces get four tigerfs-schema companion tables (`_history`, `_log`, `_savepoint`, `_metadata`) plus workspace-level dot directories `.history/`, `.log/`, `.savepoint/`, and `.undo/`. A PostgreSQL BEFORE trigger archives row snapshots into `_history`; TigerFS writes `_log` entries on each create/edit/rename/delete/undo. Rollback uses preview-then-`touch .apply`, with optional per-user scoping and a post-migration undo boundary enforced via `_metadata`.
## Enable history on a workspace
History is a synth feature flag, not a separate product mode. Enable it at workspace creation or on an existing app.
Write a `.build/` spec that includes `history` (with a format such as `markdown` or `txt`):
```bash
echo "markdown,history" > /mnt/db/.build/notes
```
TigerFS runs `GenerateBuildSQLWithFeatures`, creating the backing table, view comment (`tigerfs:md,history`), history hypertable + archive trigger, log hypertable, savepoint table, and metadata table.
```bash
echo "history" > /mnt/db/.build/notes
```
This uses `GenerateHistoryOnlySQL` to add history infrastructure and update the view comment.
```bash
ls /mnt/db/notes/.history/
ls /mnt/db/notes/.log/
ls /mnt/db/notes/.savepoint/
ls /mnt/db/notes/.undo/
```
History requires **TimescaleDB** (hypertables for `_history` and `_log`). It applies only to **file-first synth workspaces** with history enabled. Data-first tables have no history trigger. Writes that bypass the synth layer (for example direct edits under `.tables/`) are not logged.
Detection uses the view comment (`tigerfs:md,history`) or presence of a companion `_history` table (`ViewInfo.HasHistory`).
## Storage layout
```mermaid
flowchart TB
subgraph mount["Workspace mount (file-first)"]
files["notes/hello.md"]
hist[".history/"]
log[".log/"]
sp[".savepoint/"]
undo[".undo/"]
end
subgraph tigerfs["tigerfs schema"]
src["notes backing table"]
h["notes_history hypertable"]
l["notes_log hypertable"]
s["notes_savepoint"]
m["notes_metadata"]
end
files --> src
hist --> h
log --> l
sp --> s
undo --> l
undo --> h
undo --> s
src -->|"BEFORE trigger"| h
src -->|"logSynthOp on write"| l
m -->|"checkBoundary"| undo
```
| Table | Role |
|-------|------|
| `_history` | Full row snapshots (`version_id` UUIDv7 PK, `operation` create/edit/rename/delete) |
| `_log` | Operation metadata: `log_id`, `file_id`, `type`, `user_id`, denormalized `filename`, `version_id` → before-state |
| `_savepoint` | Named bookmarks: `name` (PK), `savepoint_id` (UUIDv7), `user_id`, `description` |
| `_metadata` | Workspace events; `history-format-migration` blocks undo before boundary |
Log entries are written by TigerFS (`logSynthOp`), not by database triggers. For edit/rename/delete/undo, `version_id` points at the history row captured immediately before the operation.
## Browse versions (`.history/`)
Per-directory `.history/` lists files that have snapshots at that level. Subdirectory history is scoped by `parent_id` (ADR-017 parent-pointer model).
| Path | Returns |
|------|---------|
| `workspace/.history/` | Filenames with history (+ `.by/` at workspace root) |
| `workspace/.history/file.md/` | `.id` plus version IDs, newest first |
| `workspace/.history/file.md/.id` | Stable row UUID (`file_id`) |
| `workspace/.history/file.md/` | Full synthesized file at that version |
| `workspace/.history/.by//` | Same versions keyed by row UUID (rename-safe) |
Version directory names use the UUIDv7 display form: `2026-04-07T143000.123Z-zzz0063hd8e5r42` (UTC ms timestamp + base36 entropy). Listing sorts chronologically; export JSON still uses standard hex UUIDs.
Cross-rename tracking: `file_id` is stable; history follows the row even when the leaf `filename` changes.
## Operation log (`.log/`)
`.log/` is data-first on `tigerfs._log`. Standard pipeline paths work: `.last/N`, `.first/N`, `.by//`, `.filter/`, `.order/`, `.export/json`, format suffixes.
```bash
cat /mnt/db/notes/.log/.last/10/.export/json
ls /mnt/db/notes/.log/.by/user_id/agent-7/.last/5/
ls /mnt/db/notes/.log/.by/file_id//
ls /mnt/db/notes/.log/.by/type/edit/.last/10/
```
Atomic editors (vim, VS Code, many agent write tools) often replace files via temp + rename, producing **2–3 log entries per logical save**. Undo a group by anchoring at the log entry before the group, or use `.undo/to-id//`.
### Diff symlinks
Each log entry directory exposes:
| Symlink | Target |
|---------|--------|
| `before` | `.history//` from this entry's `version_id`; `/dev/null` for `create` |
| `after` | Next log entry's before-state, or live file |
| `current` | Live workspace path; `/dev/null` if deleted |
```bash
diff -u --color /mnt/db/notes/.log//before /mnt/db/notes/.log//after
```
`filename` in the log is the **full path at operation time** (historically correct). It is not a `.log/.by/` index column — use `.history//.id` then `.log/.by/file_id//`.
## Savepoints (`.savepoint/`)
Named bookmarks in the operation timeline. Creating a savepoint does not change workspace files; deleting a savepoint does not remove log or history data.
```bash
echo '{"description":"Before refactoring"}' > /mnt/db/notes/.savepoint/before-refactor.json
cat /mnt/db/notes/.savepoint/before-refactor/description
ls /mnt/db/notes/.savepoint/.last/5/
```
Writes require a **format suffix** (`.json`, `.tsv`, `.csv`, `.yaml`). TSV uses a `description` column; JSON can be `{}` for a minimal bookmark.
| Field | Source |
|-------|--------|
| `name` | Filename without suffix (primary key) |
| `savepoint_id` | UUIDv7 at insert time (undo ordering: `log_id > savepoint_id`) |
| `user_id` | Mount identity when set |
| `description` | User text or auto-savepoint message |
Rename with `mv` updates `name` only; `savepoint_id` is unchanged, so `.undo/to-savepoint//` references remain valid.
## Auto-savepoints
Before each logged write, TigerFS compares wall-clock time to the last write on that workspace (`schema.table` key). If the gap exceeds the configured interval, it inserts an auto-savepoint **before** the new log entry.
Naming:
- `auto--20260408T143000Z` when `--user-id` / `TIGERFS_USER_ID` is set
- `auto-20260408T143000Z` for anonymous mounts
Inactivity threshold. Set to `0` to disable.
| Source | Key / flag |
|--------|------------|
| Config file | `auto_savepoint_interval: 30m` |
| Environment | `TIGERFS_AUTO_SAVEPOINT_INTERVAL=30m` |
| CLI | `--auto-savepoint-interval 30m` |
Auto-savepoints are skipped on the **first write after mount** (no prior session to separate).
## User identity and per-user undo
Identity is a single mount-level string used for new log and savepoint rows.
| Precedence | Source |
|------------|--------|
| 1 | `--user-id` |
| 2 | `TIGERFS_USER_ID` |
| 3 | Anonymous (`NULL` in log) |
```bash
cat /mnt/db/.info/user # read
echo "agent-9" > /mnt/db/.info/user # change mid-session
```
Use **separate mounts per agent** on the same database when possible. Each mount gets its own `user_id` and in-memory state; PostgreSQL MVCC provides cross-mount consistency.
Per-user undo adds `/.by/user_id//` under a `.undo/to-savepoint/` or `.undo/to-id/` path before `.apply`. Only that user's operations after the anchor are reversed; other users' log entries are ignored in the undo query.
If two users interleave edits on the **same file**, per-user undo still rolls the file back to the state before that user's first post-anchor operation — which can revert the other user's interleaved changes on that file (rollback semantics).
## Undo (`.undo/`)
Preview-then-apply: inspect `.info/summary` and the preview tree (or `.log/` symlinks for single ops), then `touch` `.apply`. The full undo runs in one PostgreSQL transaction.
### Modes
| Mode | Path | Effect |
|------|------|--------|
| Single operation | `.undo/id//` | Reverse one log entry (one file) |
| To log entry | `.undo/to-id//` | Reverse all ops **after** that `log_id` per file |
| To savepoint | `.undo/to-savepoint//` | Reverse all ops **after** `savepoint_id` per file |
`` accepts raw UUID or display name (`2026-04-08T143015.234Z-...`).
Default listing under `.undo/id/`, `.undo/to-id/`, and `.undo/to-savepoint/` is **100** most recent targets (`undo_list_limit` / `--undo-list-limit`).
### Preview and apply
```bash
cat /mnt/db/notes/.undo/to-savepoint/before-refactor/.info/summary
diff -ru /mnt/db/notes/.undo/to-savepoint/before-refactor /mnt/db/notes/ -x '.*'
touch /mnt/db/notes/.undo/to-savepoint/before-refactor/.apply
```
Multi-file preview trees include only affected paths. Files whose first post-anchor operation was `create` appear as symlinks to `/dev/null` (will be removed on apply). Restored files show synthesized content from history (`version_id`).
`.undo/id//` has **no** preview tree — use `.log//before` and `after` for diffs.
### Pipeline + `.apply`
Filters on `.undo/...` paths narrow what gets undone; preview matches apply scope. `.apply` works with `.all/`, `.last/N`, `.first/N`, `.by/`, `.filter/`, but **not** with `.sample/` (returns `EINVAL`).
Examples:
```bash
touch /mnt/db/notes/.undo/to-savepoint/x/.filter/type/delete/.apply
touch /mnt/db/notes/.undo/to-savepoint/x/.by/user_id/agent-7/.apply
touch /mnt/db/notes/.undo/to-savepoint/x/.last/5/.apply
```
### Semantics
- **Rollback**, not surgical revert: restores full row state from history.
- **Undo of undo**: undo ops are logged as `type=undo`; undoing them uses the same paths (with `undo-of-undo` dispatch via history `operation`).
- **Idempotent** undo-to-savepoint: repeating apply toward the same anchor yields the same data state (new log/history rows each time).
On success, TigerFS logs `undo applied` at Info with file counts; caches (`stat`, `path`, `undo`) invalidate under the **user schema** (where the view lives).
## Migration undo boundary
Workspaces upgraded from pre–parent-pointer history (v0.6 → v0.7) receive one `history-format-migration` row in `_metadata` when `tigerfs migrate` runs. Pre-migration history kept **lossy `parent_id`** (filled from current source, not historical placement), so undo of old edits could restore files to the wrong directory.
| Surface | Pre-boundary behavior |
|---------|----------------------|
| `.history/`, `.log/` | Read/browse still works |
| `.undo/` | Refused with **EPERM** (`ErrPermission`) when target `log_id` or `savepoint_id` sorts **before** the marker `entry_id` |
The refusal message includes the marker's `description` as the structured hint (check stderr / JSON logs on FUSE/NFS).
Fresh installs with v0.7 history have an empty metadata table — no boundary, undo works for all entries.
Verify migration and boundary behavior with `tigerfs migrate --describe` / `--dry-run` and integration coverage in `test/integration/migrate_test.go` (`verifyMigrationUndoBoundary`).
## Configuration reference
| Setting | Default | Purpose |
|---------|---------|---------|
| `user_id` / `--user-id` / `TIGERFS_USER_ID` | empty | Attributed log/savepoint rows |
| `auto_savepoint_interval` | `30m` | Session gap auto-savepoint (`0` = off) |
| `undo_list_limit` | `100` | Default `.undo/` target listing |
## Limitations and operations notes
| Topic | Constraint |
|-------|------------|
| Database | TimescaleDB hypertables + compression (7-day chunks) |
| Scope | File-first synth with `history` only |
| DDL | Undo does not reverse schema changes |
| Storage | Every edit adds a history row (compression mitigates) |
| Concurrency | Last writer wins; undo can overwrite concurrent edits |
| Direct table access | `.tables/` writes bypass logging |
Reserved workspace-level names (cannot be used as file names): `.history`, `.log`, `.savepoint`, `.undo`, plus standard capability dirs.
## Quick reference
| Goal | Command / path |
|------|----------------|
| Enable history | `echo "markdown,history" > .build/` |
| List versions | `ls /.history//` |
| Row UUID | `cat /.history//.id` |
| Recent ops | `cat /.log/.last/10/.export/json` |
| Diff one change | `diff -u /.log//before /.log//after` |
| Create savepoint | `echo '{"description":"..."}' > /.savepoint/name.json` |
| Preview undo | `cat /.undo/to-savepoint/name/.info/summary` |
| Apply undo | `touch /.undo/to-savepoint/name/.apply` |
| Per-user undo | `touch /.undo/to-savepoint/name/.by/user_id/X/.apply` |
| Set user | `echo "agent-7" > /.info/user` |
## Related pages
Create workspaces, markdown/plaintext I/O, and `.tables/` backing access.
Pipeline grammar used by `.log/` and `.savepoint/` listings.
`migrate --describe` / `--dry-run`, history-format upgrades, and boundary verification.
Why undo invalidates stat/path caches and never caches read content.
Mapping `ErrPermission` / `EINVAL` from `.undo/` to errno and zap hints.
---
## 13. Agent skills
> Bundled skills/tigerfs pack, install-time staging to agent config dirs, SKILL.md navigation (files, data, ops, recipes), and mount-time copy behavior.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/13-agent-skills.md
- Generated: 2026-06-03T07:57:27.235Z
### Source Files
- `skills/tigerfs/SKILL.md`
- `skills/tigerfs/files.md`
- `skills/tigerfs/data.md`
- `skills/tigerfs/ops.md`
- `scripts/install.sh`
- `scripts/test-install.sh`
---
title: "Agent skills"
description: "Bundled skills/tigerfs pack, install-time staging to agent config dirs, SKILL.md navigation (files, data, ops, recipes), and mount-time copy behavior."
---
TigerFS ships a portable `skills/tigerfs/` instruction pack (open `SKILL.md` format) that teaches coding agents how to use mounted PostgreSQL as files. Release archives bundle the pack via Goreleaser; `scripts/install.sh` copies it into detected agent config directories or stages it under `~/.config/tigerfs/skills/tigerfs/`. The `tigerfs mount` command does not install skills—installation runs at binary install time (or via manual copy from a clone).
## Bundled pack layout
The repository maintains five markdown files under `skills/tigerfs/`:
| File | Role |
|------|------|
| `SKILL.md` | Entry skill: mode detection, mount layout, safe editing, quick reference, anti-patterns, links to detail files |
| `files.md` | File-first: `.build/`, frontmatter, `.tables/`, history, log, savepoints, undo |
| `data.md` | Data-first: `.info/`, pipeline paths, PATCH writes, import/export, DDL staging |
| `ops.md` | CLI: `mount`, `create`, `fork`, `status`, `unmount`, config |
| `recipes.md` | Workflow and application patterns (kanban, knowledge base, session context, multi-agent undo) |
`SKILL.md` frontmatter registers the skill for agent loaders:
```yaml
name: tigerfs
description: How to discover, read, write, and search data in a TigerFS-mounted PostgreSQL database using file tools.
```
Skill packs are plain files—no model provider or hosted service is required. Any agent that loads `SKILL.md` from a filesystem or catalog path can use the same pack.
## Release bundling
Goreleaser includes the pack in every release archive:
```yaml
files:
- skills/tigerfs/*
```
After `curl -fsSL https://install.tigerfs.io | sh`, the extracted tarball contains `skills/tigerfs/` beside the `tigerfs` binary. `install.sh` runs skill installation immediately after copying the binary.
## Install-time copy and staging
`scripts/install.sh` implements all automated skill placement. Flow:
```mermaid
sequenceDiagram
participant User
participant install_sh as install.sh
participant Archive as release tar.gz
participant Stage as ~/.config/tigerfs/skills
participant Agent as agent skills dir
User->>install_sh: curl | sh
install_sh->>Archive: extract tigerfs + skills/tigerfs
install_sh->>install_sh: install_skills(skills_src)
alt stdout is TTY or TIGERFS_INTERACTIVE=1
install_sh->>User: agent menu (read /dev/tty)
User->>install_sh: 1..N, a, or s
opt choice a or N
install_sh->>Agent: copy_skills_to_agent (rm -rf tigerfs, cp -r)
opt choice s or no agents
install_sh->>Stage: stage_skills
end
else piped / non-interactive
install_sh->>Stage: stage_skills
install_sh->>User: print cp -r hints for detected agents
end
```
### Interactive vs non-interactive
Runs `interactive_install_skills` when stdout is a terminal (`-t 1`) **or** `TIGERFS_INTERACTIVE=1`. Otherwise calls `stage_skills` only.
| Mode | Trigger | Result |
|------|---------|--------|
| Interactive | TTY or `TIGERFS_INTERACTIVE=1` | Menu: install to one/all detected agents, skip (stage only), or stage when none detected |
| Non-interactive | `curl … \| sh` with piped stdin | Skills copied to `~/.config/tigerfs/skills/tigerfs/`; script prints `cp -r` commands per detected agent |
Interactive prompts read from `/dev/tty` so `curl | sh` can still ask questions without consuming the install pipe. Tests force interactivity with `TIGERFS_INTERACTIVE=1` and a fake stdin file (`scripts/test-install.sh`).
### `copy_skills_to_agent` behavior
For each selected agent:
1. `rm -rf "$agent_skills_dir/tigerfs"` — full replace on upgrade (removes stale files such as retired skill filenames).
2. `mkdir -p "$agent_skills_dir"`
3. `cp -r "$src" "$agent_skills_dir/tigerfs"`
Expected files after install (asserted in `scripts/test-install.sh`): `SKILL.md`, `files.md`, `data.md`, `recipes.md`, `ops.md`.
### Staging directory
`stage_skills` writes the same tree to:
```
~/.config/tigerfs/skills/tigerfs/
```
If agent directories exist under `$HOME`, the script prints per-agent copy lines, for example:
```bash
cp -r ~/.config/tigerfs/skills/tigerfs ~/.claude/skills/tigerfs
```
If no agents are detected, it prints a generic example (`~/.claude/skills/`).
README and CHANGELOG mention skills “automatically installed at mount time.” The `tigerfs mount` implementation (`internal/tigerfs/cmd/mount.go`) has no skill-copy hook—only `scripts/install.sh` performs automated installation today.
## Supported coding agents
Detection checks for a marker directory under `$HOME`. Installation targets the agent-specific skills path:
| Agent | Detection dir | Install path (under `$HOME`) |
|-------|---------------|------------------------------|
| Claude Code | `.claude` | `.claude/skills/tigerfs/` |
| Cursor | `.cursor` | `.cursor/skills/tigerfs/` |
| Codex CLI | `.codex` | `.agents/skills/tigerfs/` |
| Gemini CLI | `.gemini` | `.gemini/skills/tigerfs/` |
| Windsurf | `.codeium/windsurf` | `.codeium/windsurf/skills/tigerfs/` |
| Antigravity | `.gemini/antigravity` | `.gemini/antigravity/skills/tigerfs/` |
| Kiro | `.kiro` | `.kiro/steering/tigerfs/` |
Menu choices: `a` = all detected, `s` = skip (stage only), `1`…`N` = one detected agent (display order, not fixed agent index).
## SKILL.md navigation model
`SKILL.md` is the hub agents load first. It branches on **mode** and **task**, then defers to sibling files.
### Mode routing
| Signal on path | Mode | Detail file |
|----------------|------|-------------|
| `.md` / `.txt` files, workspace layout | File-first | [files.md](files.md) |
| `.info/` at table root | Data-first | [data.md](data.md) |
### Task routing from SKILL.md
| User intent | Routed to |
|-------------|-----------|
| Workspaces, frontmatter, history, undo paths | `files.md` |
| Row/column access, pipelines, bulk export, DDL | `data.md` |
| Kanban, knowledge base, session context, safe exploration | `recipes.md` (Recipes 1–7; SKILL.md maps intents to recipe numbers) |
| Mount, create, fork, cloud backends | `ops.md` |
### Cross-cutting sections in SKILL.md
- **Directory structure** — file-first tree (`.history`, `.log`, `.savepoint`, `.undo`, `.tables/`, `.build/`).
- **Safe editing** — savepoint JSON, preview via `.undo/…/.info/summary`, apply via `touch …/.apply`, multi-entry atomic-rename log groups.
- **Quick reference** — tool-oriented table (`Read`, `Write`, `Glob`, `Grep`, `Bash`).
- **Directory scanning safety** — never recurse `.log/`, `.by/`, `.export/`, etc.; use targeted paths.
- **Anti-patterns** — e.g. do not use `Write`/`Edit` to revert content; use `.undo/`; kanban uses directories + `mv`, not `status` frontmatter.
Detail files link back to SKILL.md for shared workflows (e.g. `files.md` → “Common Workflows” for undo steps).
## Install paths compared
Binary + `skills/tigerfs/` from archive; `install.sh` stages or copies to agent dirs.
Pack lives at `skills/tigerfs/` in the repo. No automatic copy to `~/.claude/skills`—point the agent at the repo path or run `cp -r skills/tigerfs ~/.claude/skills/tigerfs` (or equivalent).
Mounts PostgreSQL only; does not update agent skill directories.
Run `curl -fsSL https://install.tigerfs.io | sh` (interactive TTY shows the agent menu), or copy from `~/.config/tigerfs/skills/tigerfs/` after a non-interactive install.
Confirm `SKILL.md`, `files.md`, `data.md`, `recipes.md`, and `ops.md` exist under your agent’s `tigerfs/` skill directory.
`tigerfs mount CONNECTION MOUNTPOINT` — agents apply loaded skills when operating on the mount path.
Optional verification for the install script itself:
```bash
./scripts/test-install.sh
```
## Portable integration (BYOC/BYOK)
- **Source of truth**: markdown in `skills/tigerfs/`, not embedded in the `tigerfs` binary.
- **Distribution**: release tarball, git tree, or staged `~/.config/tigerfs/skills/tigerfs/`.
- **Agent binding**: filesystem copy into each product’s skills folder (or symlink). Grok-Wiki and other doc generators can treat the pack as a repository-local catalog entry without tying to one LLM vendor.
Reload or restart the agent session after upgrading skill files so new instructions (e.g. undo log grouping) take effect.
## Related pages
curl installer, platform prerequisites, and first binary install.
First mount and exploration after skills are in place.
Kanban, knowledge base, and session patterns from recipes.md.
Workspaces, frontmatter, and .tables/ backing access.
.history, .log, .savepoint, and .undo behavior the skills teach.
mount, create, fork, and other commands documented in ops.md.
---
## 14. Cloud backends
> tiger: and ghost: prefix resolution, create/fork/list flows, default_backend config, and Tiger CLI credential requirements.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/14-cloud-backends.md
- Generated: 2026-06-03T07:58:08.451Z
### Source Files
- `internal/tigerfs/backend/resolve.go`
- `internal/tigerfs/backend/tiger.go`
- `internal/tigerfs/backend/ghost.go`
- `internal/tigerfs/cmd/create.go`
- `internal/tigerfs/cmd/fork.go`
- `docs/adr/013-backend-prefix-scheme.md`
---
title: "Cloud backends"
description: "tiger: and ghost: prefix resolution, create/fork/list flows, default_backend config, and Tiger CLI credential requirements."
---
TigerFS connects to Tiger Cloud and Ghost by shelling out to their CLIs (`tiger`, `ghost`) through `internal/tigerfs/backend`. A `tiger:ID` or `ghost:ID` reference is parsed by `backend.Resolve()`, the backend returns a `postgres://` connection string, and mount/create/fork commands use that string while the mount registry stores `cli_backend` and `service_id` for later `info` and `fork` lookups.
## Prefix scheme
All cloud-backed references use an explicit prefix (ADR-013). `backend.Resolve()` is the single parser:
| Input pattern | `Resolve()` result | Caller behavior |
|---------------|-------------------|-----------------|
| `tiger:ID` | `TigerBackend`, ID after colon | Shell out to `tiger` CLI |
| `ghost:ID` | `GhostBackend`, ID after colon | Shell out to `ghost` CLI |
| `tiger:` or `ghost:` (empty ID) | Backend + empty ID | Auto-generate name (create/fork only) |
| `postgres://...` | `nil` backend, full URI | Direct PostgreSQL connection |
| Any other string | `nil` backend, bare string | `create`/`fork` use `default_backend`; `mount` does **not** |
**Path vs name:** Only arguments that **start with** `.` or `/` are filesystem paths (`isPath()` in fork; mount arg 2 is always a path). Names may contain slashes (e.g. `project/my-db`) without being treated as paths.
`mount` does not apply `default_backend` to bare IDs. You must use `tiger:ID`, `ghost:ID`, or `postgres://...`. A single argument without a prefix is interpreted as a **mountpoint** (connection from config/env), not a cloud service ID.
```text
User ref backend.Resolve() Next step
--------- ----------------- ---------
tiger:abc123 --> TigerBackend, "abc123" tiger db connection-string ...
ghost:mydb --> GhostBackend, "mydb" ghost connect ...
postgres://h/db --> nil, full URI connect directly
my-db (create) --> nil, "my-db" ForName(default_backend)
```
## Backend interface
`internal/tigerfs/backend` defines one interface implemented by `TigerBackend` and `GhostBackend`:
| Method | Purpose |
|--------|---------|
| `GetConnectionString(ctx, id)` | Resolve service ID → `postgres://` (with credentials) |
| `GetInfo(ctx, id)` | Live service metadata |
| `Create(ctx, opts)` | Provision new service |
| `Fork(ctx, sourceID, opts)` | Clone existing service |
`ForName("tiger"|"ghost")` maps config/registry `cli_backend` values to implementations. Empty name returns `ErrNoBackend`; unknown names error.
Credentials are **not** stored in TigerFS config for cloud mounts. Each operation invokes the backend CLI; connection strings are used immediately and registry entries store **sanitized** database URLs only.
## Tiger Cloud vs Ghost
| Behavior | Tiger CLI | Ghost CLI |
|----------|-----------|-----------|
| Connection string | `tiger db connection-string --with-password` | `ghost connect ` (password from `~/.pgpass`) |
| Service info | `tiger service get --output json` | `ghost list --json` (filter by ID; no per-ID get) |
| Create | `tiger service create [--name ] --output json --with-password` | `ghost create [--name ] --wait --json`, then `ghost connect` |
| Fork | `tiger service fork --now [--name ] --output json --with-password` | `ghost fork [--name ] --wait --json`, then `ghost connect` |
| Wait behavior | Waits by default (~30m); TigerFS does not pass `--no-wait` | Requires explicit `--wait` (TigerFS always passes it) |
| Fork strategy | Always `--now` | No strategy flag (current state) |
| Create/fork conn string | Included in JSON when `--with-password` | Fetched in a second `ghost connect` call |
TigerFS does not validate service names; the backend CLI owns format and error messages.
## Mount flow
```mermaid
sequenceDiagram
participant User
participant Mount as tigerfs mount
participant Resolve as backend.Resolve
participant BE as TigerBackend / GhostBackend
participant CLI as tiger / ghost CLI
participant Reg as mounts.json registry
User->>Mount: tiger:ID [MOUNTPOINT]
Mount->>Resolve: Resolve(connStr)
Resolve-->>Mount: Backend, serviceID
Mount->>BE: GetConnectionString(ctx, serviceID)
BE->>CLI: connection-string / connect
CLI-->>BE: postgres://...
BE-->>Mount: connStr
Mount->>Mount: mountFilesystem(connStr)
Mount->>Reg: Register(cli_backend, service_id)
```
**Arguments:**
- Two args: `CONNECTION` + `MOUNTPOINT` (connection may be `tiger:`, `ghost:`, or `postgres://`).
- One arg with prefix: connection ref; mountpoint defaults to `{default_mount_dir}/{id}` (default `/tmp`).
On success, `registerMount()` writes `service_id` and `cli_backend` (`tiger` or `ghost`) to the mount registry for `info`, `fork`, and `status`.
## Create flow
```bash
tigerfs create [BACKEND:]NAME [MOUNTPOINT] [--no-mount] [--json]
```
| `NAME` argument | Backend | Service name |
|-----------------|---------|--------------|
| `tiger:my-db` | Tiger | `my-db` |
| `ghost:my-db` | Ghost | `my-db` |
| `my-db` | `default_backend` | `my-db` |
| `tiger:` or omitted (with default) | From prefix/config | Auto-generated |
After `b.Create()`:
- Without `--no-mount`: spawns detached `tigerfs mount {cli}:{serviceID} {mountpoint}` (`startMountProcess`).
- Mountpoint: explicit arg, else `{default_mount_dir}/{name}`.
- `--no-mount`: prints `tigerfs mount {cli}:{id} /path/...` hint.
Create the cloud service but do not start a background mount process.
Emit create result including `backend`, `connection_string`, and `no_mount`.
## Fork flow
```bash
tigerfs fork SOURCE [DEST] [--name NAME] [--no-mount] [--json]
```
**SOURCE resolution (first argument):**
| SOURCE form | Resolution |
|-------------|------------|
| Path (starts with `.` or `/`) | Mount registry lookup → `cli_backend` + `service_id` |
| `tiger:ID` / `ghost:ID` | `backend.Resolve()` |
| Bare name | `default_backend` + name as service ID |
| `postgres://` mount (no cloud metadata) | Error: cannot fork — no cloud service |
**DEST resolution (optional second argument):**
| DEST form | Service name | Mountpoint |
|-----------|--------------|------------|
| Path | `--name` or basename of path | Absolute path |
| Bare name | DEST (or `--name`) | `{baseDir}/{name}` |
| Omitted | Auto (CLI) | After fork: `{baseDir}/{result.Name}` |
`baseDir` is `default_mount_dir`, or the parent directory of the source mountpoint when SOURCE was a path.
Fork requires a cloud-backed source. Raw `postgres://` mounts have empty `cli_backend` in the registry and cannot be forked through TigerFS.
## List, status, and info
| Command | Scope |
|---------|--------|
| `tigerfs list` | Active **local** mountpoints (one per line, scripting) |
| `tigerfs status` | Human-readable mount registry (PID, database, backend, uptime) |
| `tigerfs info [MOUNTPOINT]` | Registry entry + live `GetInfo()` from backend CLI when `cli_backend` and `service_id` are set |
TigerFS does not list cloud services globally. Use `tiger` / `ghost` CLIs for provider-side inventory; Ghost `GetInfo` internally runs `ghost list --json` and filters by ID.
## default_backend configuration
Cloud backend for **bare names** on `create` and `fork` only. Values: `tiger`, `ghost`, or empty (unset).
Base directory for auto-derived mountpoints. Default: `/tmp`.
```yaml
# ~/.config/tigerfs/config.yaml
default_backend: tiger # or ghost
default_mount_dir: /tmp
```
Environment: `TIGERFS_DEFAULT_BACKEND`, `TIGERFS_DEFAULT_MOUNT_DIR`.
| Scenario | Backend source |
|----------|----------------|
| `tigerfs create tiger:my-db` | `tiger:` prefix |
| `tigerfs create my-db` | `default_backend` (error if unset) |
| `tigerfs fork tiger:abc` | `tiger:` prefix |
| `tigerfs fork /mnt/prod` | Registry `cli_backend` on that mount |
| `tigerfs mount tiger:ID` | Prefix (not `default_backend`) |
| `tigerfs mount postgres://...` | Direct (no backend) |
There is no `--backend` flag; the prefix (or config for bare names on create/fork) is the only selector.
## CLI prerequisites and credentials
### Install CLIs
| Backend | Install hint (from TigerFS errors) |
|---------|-------------------------------------|
| Tiger | https://github.com/timescale/tiger-cli |
| Ghost | https://ghost.build |
TigerFS checks `exec.LookPath` before every backend call. Missing binary → `ErrCLINotFound` with install URL.
### Authenticate
TigerFS delegates auth to the backend CLI. On auth-related stderr (`not authenticated`, `login`, `unauthorized`, `401`), errors map to `ErrNotAuthenticated` with:
- Tiger: `tiger auth login`
- Ghost: `ghost auth login` (TigerFS message; Ghost docs may use `ghost login`)
**Tiger Cloud — interactive:**
```bash
tiger auth login
```
**Tiger Cloud — headless / CI (Tiger CLI reads these; bind via config or env):**
```bash
export TIGER_PUBLIC_KEY=
export TIGER_SECRET_KEY=
export TIGER_PROJECT_ID=
tiger auth login
```
Config equivalents: `tiger_public_key`, `tiger_secret_key`, `tiger_project_id` in `~/.config/tigerfs/config.yaml` (passed through Viper; used by `tiger auth login`, not stored as DB passwords by TigerFS).
**Ghost:**
- Run Ghost login flow so `ghost connect` succeeds.
- Ensure `~/.pgpass` contains credentials Ghost expects (Ghost does not use Tiger’s `--with-password`).
### Legacy config mount path
If `mount` is invoked with **no** connection argument (mountpoint only), `db.ResolveConnectionString()` may use legacy `tiger_service_id` / `TIGER_SERVICE_ID` to call `TigerBackend.GetConnectionString()`. Prefer explicit `tiger:ID` on the command line for new workflows.
## Mount registry
Cloud-backed mounts persist metadata in `~/.config/tigerfs/mounts.json` (XDG paths on other platforms):
| Field | Meaning |
|-------|---------|
| `service_id` | Tiger or Ghost service/database ID |
| `cli_backend` | `tiger` or `ghost` |
| `database` | Sanitized connection string (no password) |
| `auto_created` | TigerFS created mountpoint dir (removed on unmount) |
`fork` from a mountpoint path and `info` depend on these fields. Direct `postgres://` mounts leave `cli_backend` and `service_id` empty.
## Error handling
| Condition | Sentinel / message |
|-----------|-------------------|
| CLI not in PATH | `ErrCLINotFound` + install URL |
| Auth failure | `ErrNotAuthenticated` + `run ' auth login' first` |
| No backend for bare create/fork | `no backend specified` + config hint |
| Fork non-cloud mount | `cannot fork — filesystem has no cloud service` |
| Service missing | `service not found` (from CLI stderr) |
| Other CLI failure | `{cli} CLI error: ...` (stderr preserved) |
Passwords never land in TigerFS YAML for cloud backends. Registry and logs use `db.SanitizeConnectionString()`.
## Command quick reference
```bash title="Mount cloud service"
tigerfs mount tiger:SERVICE_ID /mnt/prod
tigerfs mount ghost:DB_ID # mountpoint → /tmp/DB_ID
```
```bash title="Create and auto-mount"
tigerfs create tiger:my-project
tigerfs create ghost:my-project /mnt/dev
tigerfs create my-project # needs default_backend
tigerfs create tiger: --no-mount
```
```bash title="Fork"
tigerfs fork /mnt/prod my-fork
tigerfs fork tiger:abc123 /mnt/staging
tigerfs fork /mnt/prod --name experiment --no-mount
```
```bash title="Inspect"
tigerfs list
tigerfs status
tigerfs info /mnt/prod
tigerfs info --json
```
## Related pages
First mount with `postgres://` or `tiger:` and success signals.
Full cobra command surface for mount, create, fork, list, status, and info.
`default_backend`, `default_mount_dir`, Tiger env vars, and precedence.
`postgres://` URIs, TLS, and password resolution after the CLI returns a conn string.
Auth errors, stale mounts, and errno recovery.
---
## 15. DDL staging
> .create/.modify/.delete staging dirs, sql/.test/.commit control files, index DDL via .indexes/, grace period, and errno on validation failures.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/15-ddl-staging.md
- Generated: 2026-06-03T07:58:30.850Z
### Source Files
- `internal/tigerfs/fs/ddl.go`
- `internal/tigerfs/fs/staging.go`
- `internal/tigerfs/fuse/control_files.go`
- `docs/adr/003-ddl-staging-pattern.md`
- `test/integration/ddl_test.go`
- `docs/spec.md`
---
title: "DDL staging"
description: ".create/.modify/.delete staging dirs, sql/.test/.commit control files, index DDL via .indexes/, grace period, and errno on validation failures."
---
TigerFS applies schema changes through in-memory DDL staging sessions exposed as directories under `/.create/`, `//.modify/`, `//.delete/`, and related paths. Each session holds staged SQL plus trigger files (`sql`, `.test`, `test.log`, `.commit`, `.abort`); `fs.DDLManager` validates with `BEGIN`/`ROLLBACK` and commits with a real `Exec`, while FUSE and NFS adapters map `fs.FSError` codes to POSIX errno and log hints to stderr.
DDL staging is not row staging. `StagingManager` in `internal/tigerfs/fs/staging.go` tracks partial `INSERT`s when you `mkdir` a new primary-key directory under a table. DDL staging uses `DDLManager` in `internal/tigerfs/fs/ddl.go` and only affects schema objects.
## Supported operations and paths
All DDL types share the same control-file workflow (ADR-003). Path resolution is `PathDDL` in `internal/tigerfs/fs/path.go`.
| Object | Create | Modify | Delete |
|--------|--------|--------|--------|
| Table | `/.create//` | `//.modify/` | `//.delete/` |
| Index | `//.indexes/.create//` | — | `//.indexes//.delete/` |
| Schema | `/.schemas/.create//` | — | `/.schemas//.delete/` |
| View | `/.views/.create//` | — | `/.views//.delete/` |
Existing indexes also expose read-only metadata at `//.indexes//.schema` (generated `CREATE INDEX` DDL, not a staging directory).
```text
/mnt/db/
├── .create/orders/ sql .test test.log .commit .abort
├── users/
│ ├── .modify/ (auto-session on first access)
│ ├── .delete/
│ └── .indexes/
│ ├── .create/email_idx/
│ └── email_idx/.delete/
├── .schemas/.create/app/
└── .views/.create/active_users/
```
## Control files
| File | Type | Read | Write / touch |
|------|------|------|----------------|
| `sql` | Content | Staged SQL, or generated template if empty | Stage DDL; resets validation |
| `.test` | Trigger | Empty | Run `ExecInTransaction` (rolled back) |
| `test.log` | Content | Last validation output | Read-only (`ENOENT` until first test) |
| `.commit` | Trigger | Empty | Execute staged SQL |
| `.abort` | Trigger | Empty | Mark session completed (cancel) |
Trigger files must be opened for **write** (for example `touch` or `os.Create`), not read-only open — integration tests document that read-only open does not fire the write path.
Comment stripping uses `ExtractSQL` / `IsEmptyOrCommented`: lines starting with `--` and block comments `/* … */` are removed before test or commit. Comment-only `sql` is rejected at test/commit with a message in `test.log` or commit error text.
## Workflow
For **new** root-level objects (`/.create//`, `/.schemas/.create/`, `/.views/.create/`, index create under `.indexes/.create/`), run `mkdir`:
```bash
mkdir /mnt/db/.create/orders
mkdir /mnt/db/users/.indexes/.create/email_idx
```
For **modify**, **delete**, and index delete paths on existing objects, TigerFS **auto-creates** a session on first `ls`, `stat`, or read of `sql` — no `mkdir` required.
```bash
cat /mnt/db/.create/orders/sql # template with commented examples
vi /mnt/db/.create/orders/sql # or echo/redirection
```
Writing `sql` clears the `Validated` flag. Vim/emacs swap files are stored in-memory on the session (`ExtraFiles`), not on disk.
```bash
touch /mnt/db/.create/orders/.test
cat /mnt/db/.create/orders/test.log
```
A failed validation updates `test.log` but the touch itself **succeeds** at the filesystem layer (no errno for bad SQL). Check `test.log` and stderr JSON logs (`logging.Warn` / `logging.Info`).
```bash
touch /mnt/db/.create/orders/.commit # applies DDL, invalidates metadata cache
# or
touch /mnt/db/.create/orders/.abort # cancels; sql reverts to template on read
```
On commit failure the session stays active so you can fix `sql` and retry. On success the session is marked `Completed` and kept in memory for the grace period.
Script-style one-liner (no `.test`):
```bash
mkdir /mnt/db/.create/orders && \
echo 'CREATE TABLE orders (id serial PRIMARY KEY)' > /mnt/db/.create/orders/sql && \
touch /mnt/db/.create/orders/.commit
```
## Session lifecycle and grace period
```mermaid
stateDiagram-v2
[*] --> Active: mkdir or auto-create
Active --> Active: write sql
Active --> Active: touch .test
Active --> Completed: touch .commit success
Active --> Completed: touch .abort
Completed --> Reaped: grace period elapsed
Reaped --> [*]
Active --> Active: commit failure
```
| State | Behavior |
|-------|----------|
| Active | SQL and control files fully functional |
| Completed | After successful commit or abort; trigger touches are **no-ops** |
| Reaped | Lazy removal on `ListSessionEntries` / `FindSessionByName` after grace elapsed |
How long completed sessions remain visible so NFS/FUSE post-close `stat`/`readdir` do not fail. Set in `~/.config/tigerfs/config.yaml` or `TIGERFS_DDL_GRACE_PERIOD`. Zero means use the 30s default (`config.DDLGracePeriod` → `NewDDLManager`).
Staged SQL lives **in memory per mount**. Unmount drops all sessions. This is intentional — no stale DDL on disk.
## Creating sessions: mkdir rules
| Situation | errno | Hint (stderr) |
|-----------|-------|----------------|
| Duplicate active session | `EEXIST` | Abort existing session first |
| Empty name `/.create/` | `EINVAL` | Use `mkdir /.create/` |
| Unknown operation in path | `EINVAL` | Malformed DDL path |
| Access `sql` without session (root create) | `ENOENT` | `mkdir /.create/` |
Replacing a **completed** root-level create session: `mkdir` removes the old session and starts fresh. Completed table-level auto-sessions are replaced silently on next access.
## errno and logging on validation failures
FUSE/NFS only return errno codes; details go to **stderr** via structured zap logs (`logging.Error` with `hint`).
| Operation | Condition | errno | Where detail appears |
|-----------|-----------|-------|-------------------|
| Touch `.test` | Bad SQL | *(success)* | `test.log`, stderr warn |
| Touch `.test` | No / comment-only SQL | *(success)* | `test.log` |
| Read `test.log` | Before any test | `ENOENT` | Hint: run `touch …/.test` |
| Touch `.commit` | Empty / comment-only SQL | `EIO` | stderr + error message |
| Touch `.commit` | PostgreSQL error | `EIO` | stderr; session preserved |
| Write `sql` | Session already completed | `EPERM` | Hint: new `mkdir` |
| `mkdir` duplicate active | `EEXIST` | stderr hint |
| `rm` control file or staging dir | — | `EACCES` | Use `.abort` |
| Read `test.log` open for write | — | `EACCES` | Read-only file |
Do not infer validation success from the exit code of `touch .test`. Always read `test.log` or stderr. Commit failures use `EIO` — read TigerFS logs when the shell only prints `Input/output error`.
Example stderr after failed commit:
```json
{"message":"DDL commit failed for create/orders","hint":"session=… error: …"}
```
## Index DDL
Indexes use the same control files under the owning table:
```bash
mkdir /mnt/db/users/.indexes/.create/email_idx
echo 'CREATE INDEX email_idx ON users (email)' > /mnt/db/users/.indexes/.create/email_idx/sql
touch /mnt/db/users/.indexes/.create/email_idx/.test
cat /mnt/db/users/.indexes/.create/email_idx/test.log
touch /mnt/db/users/.indexes/.create/email_idx/.commit
# Drop
echo 'DROP INDEX email_idx' > /mnt/db/users/.indexes/email_idx/.delete/sql
touch /mnt/db/users/.indexes/email_idx/.delete/.commit
```
`DDLParentTable` is set from path context so templates reference the correct table. Primary key indexes are omitted from `.indexes/` listings (row paths remain the access model).
## Implementation map
| Layer | Role |
|-------|------|
| `fs/path.go` | Parses `PathDDL`, `DDLOp`, `DDLFile`, object type |
| `fs/ddl.go` | `DDLManager`: sessions, test, commit, templates, grace reaper |
| `fs/write.go` | `mkdirDDL`, `writeDDLFile`, metadata cache invalidation on commit |
| `fs/operations.go` | `ensureDDLSession`, directory listings, stable mtimes |
| `fuse/adapter.go` | `ErrorToErrno` + hint logging |
| `fuse/control_files.go` | Legacy FUSE-only nodes when `legacy_fuse: true` |
Default mounts route through `fs.Operations`; both Linux FUSE and macOS NFS use the same DDL semantics.
## Templates
Reading `sql` before writing returns object-specific commented templates (`generateCreateTemplate`, `generateModifyTemplate`, `generateDeleteTemplate` in `ddl.go`). Templates encourage uncommenting real statements rather than executing comments.
## Verification
Integration coverage in `test/integration/ddl_test.go` exercises table create/delete, `ALTER` via `.modify`, index and schema cycles, view create/delete, valid/invalid `.test`, abort clearing staged content, and script workflow.
```bash
go test -run TestDDL_ ./test/integration/...
go test -run 'TestMkdir_PathDDL|TestWriteDDLFile' ./internal/tigerfs/fs/...
```
After create/delete, allow a short pause (or cache TTL) before `stat` on new table paths — tests sleep ~500ms for metadata visibility.
## Related pages
Path model, dot-directories, and how `fs.Operations` maps paths to SQL.
`fs.ErrorCode` to errno mapping and reading stderr hints.
`ddl_grace_period` and mount-level settings.
Build, `go test`, and integration testcontainers workflow.
---
## 16. CLI reference
> All cobra commands (mount, unmount, stop, status, info, list, create, fork, migrate, test-connection, config, version), flags, and argument patterns.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/16-cli-reference.md
- Generated: 2026-06-03T08:00:10.112Z
### Source Files
- `internal/tigerfs/cmd/root.go`
- `internal/tigerfs/cmd/mount.go`
- `internal/tigerfs/cmd/migrate.go`
- `internal/tigerfs/cmd/config.go`
- `docs/spec.md`
- `internal/tigerfs/cmd/cmd_test.go`
---
title: "CLI reference"
description: "All cobra commands (mount, unmount, stop, status, info, list, create, fork, migrate, test-connection, config, version), flags, and argument patterns."
---
The `tigerfs` binary is a Cobra application rooted at `internal/tigerfs/cmd`: `cmd.Execute` builds `buildRootCmd`, runs `PersistentPreRunE` (logging + `config.Init`), then dispatches subcommands. Mount is a blocking long-lived process; `create` and `fork` optionally spawn a detached `tigerfs mount` child. Active mounts are recorded in a JSON registry (default `~/.config/tigerfs/mounts.json`) for `status`, `list`, `info`, `unmount`, and `stop`.
## Command tree
```text
tigerfs # root: persistent logging/config flags
├── mount [CONNECTION] [MOUNTPOINT]
├── unmount MOUNTPOINT
├── stop PID
├── status [MOUNTPOINT]
├── info [MOUNTPOINT]
├── list
├── create [BACKEND:]NAME [MOUNTPOINT]
├── fork SOURCE [DEST]
├── migrate [CONNECTION]
├── test-connection [CONNECTION]
├── config
│ ├── show
│ ├── validate
│ └── path
└── version
```
`mount` is documented as the primary workflow, but the root command does not register a Cobra default command. Invoke `tigerfs mount …` explicitly (not `tigerfs /mnt/db` alone).
## Global flags
Applied on every subcommand via root `PersistentPreRunE` (flags bind to Viper at run time).
| Flag | Default | Effect |
|------|---------|--------|
| `--config-dir` | `~/.config/tigerfs` (or `XDG_CONFIG_HOME` / `%APPDATA%`) | Config search path |
| `--log-level` | `warn` | `debug`, `info`, `warn`, `error` |
| `--log-file` | (empty → stderr) | Log file path |
| `--log-format` | `text` | `text` or `json` |
| `--log-sql-params` | `false` | Log SQL bind parameters (may leak secrets) |
Configuration merge order for runtime behavior: built-in defaults → `config.yaml` → `TIGERFS_*` and bound `PG*` / `TIGER_*` env vars → command-specific flag overrides in each command’s `RunE`.
## Connection argument patterns
Many commands accept an optional `CONNECTION` positional argument. Resolution rules differ by command.
### Prefix scheme (mount, migrate)
| Form | Meaning |
|------|---------|
| `tiger:ID` | Tiger Cloud service; CLI fetches `postgres://…` via backend |
| `ghost:ID` | Ghost database; same pattern |
| `postgres://…` | Direct URL (two-arg mount: connection + mountpoint) |
| (omitted) | See per-command resolution below |
`backend.Resolve` treats only `tiger:` and `ghost:` as cloud backends. A bare name (no prefix) is not a connection string; callers use `default_backend` in config (`create`, `fork`) or treat a single non-prefixed arg as a mountpoint path (`mount`).
### `db.ResolveConnectionString` (test-connection, migrate)
When `CONNECTION` is omitted:
1. Explicit argument if provided
2. `tiger_service_id` / `TIGER_SERVICE_ID` in config
3. `host` from config (including `PGHOST`)
`mount` does **not** call `ResolveConnectionString`. It resolves `tiger:` / `ghost:` via `backend.GetConnectionString` only when the connection arg is non-empty after `resolveMountArgs`.
## `mount`
**Use:** `tigerfs mount [CONNECTION] [MOUNTPOINT]`
**Args:** `cobra.RangeArgs(1, 2)` — one or two positionals.
Blocks until SIGINT/SIGTERM (from `cmd/tigerfs/main.go` cancel) or external unmount. Registers the mount in the registry on success; unregisters on exit. Auto-creates missing mountpoint directories and removes them on unmount only when TigerFS created the directory.
### Argument resolution
| Args | `CONNECTION` | `MOUNTPOINT` |
|------|--------------|--------------|
| `CONN MNT` | First arg | Second arg |
| `tiger:ID` or `ghost:ID` | Prefix ref | `default_mount_dir/ID` (default `/tmp/ID`) |
| Single path or name | Empty | That arg (e.g. `/mnt/db`, `./db`) |
A single `postgres://…` positional is parsed as a **mountpoint**, not a connection. Use two args: `tigerfs mount postgres://user@host/db /mnt/db`.
### Mount flags (applied in `RunE`)
| Flag | Default | Maps to / behavior |
|------|---------|-------------------|
| `--schema` | `""` | `cfg.DefaultSchema` when set |
| `--no-filename-extensions` | `false` | `cfg.NoFilenameExtensions` |
| `--query-timeout` | `0` | `cfg.QueryTimeout` when > 0 |
| `--dir-filter-limit` | `0` | `cfg.DirFilterLimit` when > 0 |
| `--max-pipeline-depth` | `0` | `cfg.MaxPipelineDepth` only if flag changed |
| `--legacy-fuse` | `false` | Linux: legacy FUSE tree vs `fs.Operations` (default) |
| `--insecure-no-ssl` | `false` | `cfg.InsecureNoSSL` |
| `--user-id` | `""` | `cfg.UserID`, else `TIGERFS_USER_ID` |
| `--auto-savepoint-interval` | `0` | `cfg.AutoSavepointInterval` when non-zero; must be ≥ 0 |
| `--undo-list-limit` | `0` | `cfg.UndoListLimit` when > 0 |
### Mount flags (registered, not wired in `RunE`)
These appear in `--help` but are not passed into `cfg` or the mount layer today:
| Flag | Default | Note |
|------|---------|------|
| `--read-only` | `false` | Logged only |
| `--max-ls-rows` | `10000` | Use `dir_listing_limit` in config (default 1000) |
| `--foreground` | `false` | Mount always blocks in the foreground process |
### Platform backends
| OS | Backend |
|----|---------|
| Linux | FUSE (`fuse.MountOps` default; `--legacy-fuse` → `fuse.Mount`) |
| macOS | In-process NFS (`nfs.Mount`) |
### Examples
```bash
# Cloud: auto mountpoint under /tmp
tigerfs mount tiger:abcde12345
# Cloud + explicit path
tigerfs mount ghost:fghij67890 /mnt/db
# Direct URL (two positionals required)
tigerfs mount postgres://user@host:5432/mydb /mnt/db
# Undo identity + debug logs
tigerfs mount --user-id agent-1 --log-level debug postgres://localhost/mydb /mnt/db
```
## `unmount`
**Use:** `tigerfs unmount MOUNTPOINT`
**Args:** Exactly one mountpoint (resolved to absolute path).
| Flag | Shorthand | Default | Description |
|------|-----------|---------|-------------|
| `--force` | `-f` | `false` | Force unmount (`fusermount -uz` on Linux, `diskutil unmount force` on macOS) |
| `--timeout` | `-t` | `30` | Seconds to wait for graceful shutdown |
**Behavior:** Looks up registry → `SIGTERM` to registered PID → waits → falls back to OS unmount. Cleans registry entry and removes auto-created mountpoint dirs when applicable. Prints `Successfully unmounted `.
## `stop`
**Use:** `tigerfs stop PID`
**Args:** Positive integer PID.
| Flag | Shorthand | Default |
|------|-----------|---------|
| `--timeout` | `-t` | `30` |
Sends `SIGTERM`, waits for exit, cleans registry if the PID matches a registered mount. Equivalent to unmount when you have the PID from `status` instead of the path.
## `status`
**Use:** `tigerfs status [MOUNTPOINT]`
**Args:** Zero or one.
| Mode | Output |
|------|--------|
| No args | Table: `MOUNTPOINT`, `DATABASE`, `PID`, `STATUS`, `UPTIME` for active mounts (stale entries cleaned first) |
| With mountpoint | Detail: mountpoint, database, backend, service ID, PID, status (`active` / `stale`), started time, uptime |
Empty registry: `No active TigerFS mounts`.
## `list`
**Use:** `tigerfs list`
**Args:** None.
One mountpoint path per line (no header). Stale registry entries are cleaned before listing. Intended for scripting (`xargs`, loops).
## `info`
**Use:** `tigerfs info [MOUNTPOINT]`
**Args:** Zero or one.
| Flag | Description |
|------|-------------|
| `--json` | JSON `backend.MountInfo` |
Without `MOUNTPOINT`: requires exactly one active mount; errors if zero or multiple (lists suggested commands). For cloud-backed mounts (`CLIBackend` + `ServiceID`), calls the backend CLI for live service metadata.
## `create`
**Use:** `tigerfs create [BACKEND:]NAME [MOUNTPOINT]`
**Args:** `cobra.MaximumNArgs(2)`.
| Flag | Description |
|------|-------------|
| `--no-mount` | Create service only; print manual mount hint |
| `--json` | JSON result including `backend`, `no_mount` |
**Backend resolution:** `tiger:NAME`, `ghost:NAME`, or bare `NAME` with `default_backend` in config. Bare `tiger:` / `ghost:` auto-generates a service name.
**Mount:** Unless `--no-mount`, runs detached `tigerfs mount : `. Default mountpoint: `default_mount_dir/` (often `/tmp/`).
## `fork`
**Use:** `tigerfs fork SOURCE [DEST]`
**Args:** One or two.
| Flag | Description |
|------|-------------|
| `--name` | Override fork service name |
| `--no-mount` | Fork only |
| `--json` | JSON fork result |
**SOURCE**
| Form | Resolution |
|------|------------|
| Path starting with `/` or `.` | Mount registry lookup (requires cloud-backed mount) |
| `tiger:ID`, `ghost:ID` | Backend + service ID |
| Bare name | `default_backend` |
**DEST** (optional): path → mount there (basename used as name unless `--name`); bare name → service name and mount at `baseDir/NAME` (`baseDir` is parent of source mountpoint when SOURCE was a path, else `default_mount_dir`).
## `migrate`
**Use:** `tigerfs migrate [CONNECTION]`
**Args:** `cobra.MaximumNArgs(1)`. Uses `db.ResolveConnectionString`. 60s command timeout.
| Flag | Description |
|------|-------------|
| `--describe` | List pending migrations and items |
| `--dry-run` | Print SQL without executing |
| `--schema` | Schema to migrate (default: `current_schema()`) |
| `--insecure-no-ssl` | Disable TLS enforcement for remote hosts |
**Registered migrations (order fixed):**
| Name | Summary |
|------|---------|
| `move-backing-tables` | Move synth backing tables from `_name` in user schema to `tigerfs.name` |
| `relational-directories` | Add `parent_id` parent-pointer directory model |
| `parent-dir-mtime-trigger` | Parent directory `modified_at` trigger for NFS listings |
Default mode runs each pending migration in a transaction. Idempotent when nothing pending: `No pending migrations.`
## `test-connection`
**Use:** `tigerfs test-connection [CONNECTION]`
**Args:** `cobra.MaximumNArgs(1)`. 30s timeout. Uses `db.ResolveConnectionString`.
**Output:** PostgreSQL version (trimmed), database, user, schema count, base table count (user schemas only).
## `config`
**Use:** `tigerfs config `
### `config show`
Merged effective config as YAML (passwords omitted). Sections: `connection`, `tiger_cloud`, `backend`, `filesystem`, `query`, `metadata`, `nfs`, `ddl`, `logging`, `advanced`.
### `config validate`
Reads `~/.config/tigerfs/config.yaml` (or `--config-dir`). Missing file is OK (“Using default configuration”). Validates YAML, unmarshaling, port/pool/dir limits, `default_format`, `binary_encoding`, `log_level`.
### `config path`
Prints config file path; notes if file does not exist.
## `version`
**Use:** `tigerfs version`
Prints: `Version`, `BuildTime`, `GitCommit` (ldflags in release builds; `dev` / `unknown` in dev), Go version, `GOOS`/`GOARCH`.
## Mount lifecycle
```mermaid
stateDiagram-v2
[*] --> Starting: tigerfs mount
Starting --> Registered: register mounts.json
Registered --> Serving: fs.Serve blocks
Serving --> Unregistering: SIGTERM or unmount
Unregistering --> [*]: unregister + optional rmdir
```
## Operational commands (quick reference)
| Goal | Command |
|------|---------|
| Mount cloud DB | `tigerfs mount tiger:SERVICE_ID` |
| List mountpoints | `tigerfs list` |
| Inspect mounts | `tigerfs status` |
| Cloud service details | `tigerfs info --json /path` |
| Test credentials | `tigerfs test-connection postgres://…` |
| Graceful teardown | `tigerfs unmount /path` or `tigerfs stop PID` |
| Provision + mount | `tigerfs create tiger:my-db` |
| Clone + mount | `tigerfs fork /mnt/prod my-fork` |
| Schema upgrades | `tigerfs migrate postgres://… --dry-run` |
## Error and exit behavior
- Root and subcommands set `SilenceUsage: true` on failure (no usage spam; message on stderr via logging).
- Invalid args (bad PID, missing mount, multiple mounts for bare `info`) return descriptive errors from `RunE`.
- FUSE/NFS operational errors surface as POSIX errno to tools; detailed hints are written via `logging.Error` / `Warn` (see error-codes page).
`cmd_test.go` verifies command names, help text, and core flag registration. When `--help` disagrees with runtime wiring, trust the `RunE` implementation in `internal/tigerfs/cmd/*.go`.
## Related pages
Config file fields, env vars, and precedence beyond CLI flags.
postgres:// URIs, PG* env, TLS, passwords, and pool settings.
tiger: and ghost: resolution, create/fork, and Tiger CLI auth.
migrate --describe/--dry-run and history-format upgrade boundaries.
Stale mounts, force unmount, and --log-level debug.
---
## 17. Configuration reference
> Config struct fields, ~/.config/tigerfs/config.yaml, TIGERFS_* env vars, precedence, TLS enforcement, and mount-specific overrides.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/17-configuration-reference.md
- Generated: 2026-06-03T07:59:01.625Z
### Source Files
- `internal/tigerfs/config/config.go`
- `internal/tigerfs/config/config_test.go`
- `internal/tigerfs/cmd/config.go`
- `docs/spec.md`
- `internal/tigerfs/db/tls.go`
---
title: "Configuration reference"
description: "Config struct fields, ~/.config/tigerfs/config.yaml, TIGERFS_* env vars, precedence, TLS enforcement, and mount-specific overrides."
---
TigerFS loads a single `config.Config` value per process via Viper (`internal/tigerfs/config`), merges defaults with `~/.config/tigerfs/config.yaml` and environment variables, then applies command-specific overrides (notably on `mount`) before passing the struct into `db.NewClient` and `fs.Operations`. Use `tigerfs config show` to inspect the effective merged settings; never read Viper directly outside the `cmd` package.
## Configuration flow
```mermaid
flowchart TB
subgraph sources [Sources lowest to highest]
D[Init defaults in config.Init]
F["config.yaml in GetDefaultConfigDir"]
E["TIGERFS_* / PG* / TIGER_* env"]
R["Root persistent flags bound in PersistentPreRunE"]
end
subgraph load [Per command]
I[config.Init in root PersistentPreRunE]
L[config.Load into Config struct]
end
subgraph mount [Mount only]
M[Mount RunE patches cfg fields]
C[db.NewClient enforceSSLMode + pool]
O[fs.Operations + FUSE/NFS adapter]
end
D --> F --> E --> R --> I --> L
L --> M --> C --> O
```
`config.Init()` runs on every command via the root command’s `PersistentPreRunE`. Subcommands call `config.Load()` to obtain a `*config.Config`. Application code should use that struct, not `viper` directly.
## Precedence
Later layers override earlier ones for keys Viper manages:
| Priority | Source | Examples |
|----------|--------|----------|
| 1 (lowest) | Built-in defaults | `port: 5432`, `dir_listing_limit: 1000` |
| 2 | Config file | `~/.config/tigerfs/config.yaml` |
| 3 | Environment | `TIGERFS_LOG_LEVEL`, `PGHOST`, `TIGER_SERVICE_ID` |
| 4 | Root persistent CLI flags | `--log-level`, `--config-dir` (bound to Viper before `Init`) |
| 5 | Mount-time patches | `--schema`, `--insecure-no-ssl`, `--user-id` (applied to `cfg` after `Load`) |
| 6 (connection) | Connection string | Password in URL; `sslmode` transformed at connect |
**`TIGERFS_*` vs `PG*`:** For the same logical key (e.g. port), `TIGERFS_PORT` wins over `PGPORT` when both are set.
**Password resolution** (separate from Viper; in `db.resolvePassword`):
1. Password embedded in the connection string
2. `PGPASSWORD`
3. `TIGERFS_PASSWORD`
4. `password_command` from config
5. `~/.pgpass` (pgx automatic)
## Config file location and format
### Path resolution
| Platform / variable | Directory |
|---------------------|-----------|
| Default (Unix) | `~/.config/tigerfs/` |
| `XDG_CONFIG_HOME` set | `$XDG_CONFIG_HOME/tigerfs/` |
| Windows `APPDATA` | `%APPDATA%/tigerfs/` |
File name: `config.yaml`. A missing file is not an error.
```bash
tigerfs config path # print path; note if file missing
tigerfs config show # merged effective config (YAML, secrets omitted)
tigerfs config validate
```
`config.Init()` always calls `viper.AddConfigPath(GetDefaultConfigDir())`, which follows `XDG_CONFIG_HOME` / `APPDATA` / home — not the `config_dir` field after load. To relocate the config directory, set `XDG_CONFIG_HOME` (or `APPDATA` on Windows). The `config_dir` / `TIGERFS_CONFIG_DIR` value is stored on `Config` and shown in `config show`, but does not change where Viper searches for `config.yaml` today.
### YAML shape
Keys are **flat snake_case** matching `mapstructure` tags on `Config`. Example:
```yaml
# Connection
host: localhost
port: 5432
user: myuser
database: mydb
default_schema: public
pool_size: 10
pool_max_idle: 5
password_command: "op read op://vault/db/password"
insecure_no_ssl: false
# Backend
default_backend: tiger
default_mount_dir: /tmp
# Filesystem
dir_listing_limit: 1000
dir_writing_limit: 100000
trailing_newlines: true
no_filename_extensions: false
attr_timeout: 1s
entry_timeout: 1s
default_format: tsv
binary_encoding: raw
max_pipeline_depth: 10
# Query safety
query_timeout: 30s
dir_filter_limit: 100000
# Metadata caches (catalog vs structural)
metadata_refresh_interval: 10s
structural_metadata_refresh_interval: 5m
# NFS write buffers (macOS)
nfs_streaming_threshold: 10485760
nfs_max_random_write_size: 104857600
nfs_cache_reaper_interval: 30s
nfs_cache_idle_timeout: 5m
# DDL
ddl_grace_period: 30s
# Logging
log_level: warn
log_file: ""
log_format: text
log_sql_params: false
# Identity / undo
user_id: agent-7
auto_savepoint_interval: 30m
undo_list_limit: 100
# Linux FUSE
legacy_fuse: false
```
Do not put passwords in the config file; use `.pgpass`, env vars, or `password_command`.
## Environment variables
Viper uses prefix `TIGERFS` with `AutomaticEnv()` (e.g. `TIGERFS_DIR_LISTING_LIMIT` → `dir_listing_limit`). Duration env values accept Go duration strings (`30s`, `1m`, `5m`).
### PostgreSQL standard (bound in `config.Init`)
| Variable | Config field |
|----------|----------------|
| `PGHOST` | `host` |
| `PGPORT` | `port` |
| `PGUSER` | `user` |
| `PGDATABASE` | `database` |
| `PGPASSWORD` | (password resolution only) |
### Tiger Cloud (headless / Docker)
| Variable | Config field |
|----------|----------------|
| `TIGER_SERVICE_ID` | `tiger_service_id` |
| `TIGER_PUBLIC_KEY` | `tiger_public_key` |
| `TIGER_SECRET_KEY` | `tiger_secret_key` |
| `TIGER_PROJECT_ID` | `tiger_project_id` |
### Common `TIGERFS_*` variables
| Environment variable | Field | Default |
|---------------------|-------|---------|
| `TIGERFS_CONFIG_DIR` | `config_dir` | from `GetDefaultConfigDir()` |
| `TIGERFS_DEFAULT_SCHEMA` | `default_schema` | `""` (inherit PG `current_schema`) |
| `TIGERFS_DIR_LISTING_LIMIT` | `dir_listing_limit` | `1000` |
| `TIGERFS_DIR_WRITING_LIMIT` | `dir_writing_limit` | `100000` |
| `TIGERFS_DIR_FILTER_LIMIT` | `dir_filter_limit` | `100000` |
| `TIGERFS_QUERY_TIMEOUT` | `query_timeout` | `30s` |
| `TIGERFS_MAX_PIPELINE_DEPTH` | `max_pipeline_depth` | `10` (`0` = unlimited) |
| `TIGERFS_TRAILING_NEWLINES` | `trailing_newlines` | `true` |
| `TIGERFS_NO_FILENAME_EXTENSIONS` | `no_filename_extensions` | `false` |
| `TIGERFS_ATTR_TIMEOUT` | `attr_timeout` | `1s` |
| `TIGERFS_ENTRY_TIMEOUT` | `entry_timeout` | `1s` |
| `TIGERFS_METADATA_REFRESH_INTERVAL` | `metadata_refresh_interval` | `10s` |
| `TIGERFS_STRUCTURAL_METADATA_REFRESH_INTERVAL` | `structural_metadata_refresh_interval` | `5m` |
| `TIGERFS_DEFAULT_FORMAT` | `default_format` | `tsv` |
| `TIGERFS_BINARY_ENCODING` | `binary_encoding` | `raw` |
| `TIGERFS_POOL_SIZE` | `pool_size` | `10` |
| `TIGERFS_POOL_MAX_IDLE` | `pool_max_idle` | `5` |
| `TIGERFS_PASSWORD` | (password resolution) | — |
| `TIGERFS_PASSWORD_COMMAND` | `password_command` | — |
| `TIGERFS_DEFAULT_BACKEND` | `default_backend` | `""` |
| `TIGERFS_DEFAULT_MOUNT_DIR` | `default_mount_dir` | `/tmp` |
| `TIGERFS_INSECURE_NO_SSL` | `insecure_no_ssl` | `false` |
| `TIGERFS_USER_ID` | `user_id` | `""` |
| `TIGERFS_AUTO_SAVEPOINT_INTERVAL` | `auto_savepoint_interval` | `30m` (`0` disables) |
| `TIGERFS_UNDO_LIST_LIMIT` | `undo_list_limit` | `100` |
| `TIGERFS_LOG_LEVEL` | `log_level` | `warn` |
| `TIGERFS_LOG_FILE` | `log_file` | `""` (stderr) |
| `TIGERFS_LOG_FORMAT` | `log_format` | `text` |
| `TIGERFS_LOG_SQL_PARAMS` | `log_sql_params` | `false` |
| `TIGERFS_LEGACY_FUSE` | `legacy_fuse` | `false` |
| NFS / DDL keys | `nfs_*`, `ddl_grace_period` | see defaults table |
## `Config` struct reference
All fields live in `internal/tigerfs/config/config.go`. Grouped by concern:
### Connection and TLS
| Field | YAML / env | Default | Role |
|-------|------------|---------|------|
| `Host` | `host` / `PGHOST` | empty | Server hostname |
| `Port` | `port` / `PGPORT` / `TIGERFS_PORT` | `5432` | TCP port |
| `User` | `user` / `PGUSER` | empty | DB user |
| `Database` | `database` / `PGDATABASE` | empty | Database name |
| `Password` | — | empty | Rarely set via file; prefer env/pgpass |
| `DefaultSchema` | `default_schema` | `""` | Flatten schema to mount root |
| `PoolSize` | `pool_size` | `10` | Max pool connections |
| `PoolMaxIdle` | `pool_max_idle` | `5` | Min idle connections |
| `PasswordCommand` | `password_command` | empty | Shell command → password on stdout |
| `InsecureNoSSL` | `insecure_no_ssl` / `--insecure-no-ssl` | `false` | Skip remote TLS enforcement |
Tiger Cloud credential fields: `TigerCloudServiceID`, `TigerCloudPublicKey`, `TigerCloudSecretKey`, `TigerCloudProjectID` (env `TIGER_*`).
### Backend and mount paths
| Field | Default | Role |
|-------|---------|------|
| `DefaultBackend` | `""` | `tiger` or `ghost` for bare service IDs |
| `DefaultMountDir` | `/tmp` | Base dir when `tiger:ID` / `ghost:ID` omit mountpoint |
### Filesystem behavior
| Field | Default | Role |
|-------|---------|------|
| `DirListingLimit` | `1000` | Max rows for `ls`, `.by/`, exports without explicit limit |
| `DirWritingLimit` | `100000` | Bulk import row cap |
| `TrailingNewlines` | `true` | Append `\n` on column/count reads |
| `NoFilenameExtensions` | `false` | Disable type-based extensions (`.json`, `.txt`, …) |
| `AttrTimeout` | `1s` | FUSE attribute cache (Linux only) |
| `EntryTimeout` | `1s` | FUSE entry cache (Linux only) |
| `MaxPipelineDepth` | `10` | Hide deeper pipeline capabilities (`0` = unlimited) |
| `DefaultFormat` | `tsv` | Row-as-file default (`tsv`, `csv`, `json`) |
| `BinaryEncoding` | `raw` | BYTEA encoding (`raw`, `base64`, `hex`) |
### Query safety
| Field | Default | Role |
|-------|---------|------|
| `QueryTimeout` | `30s` | Per-connection `statement_timeout` + context timeout |
| `DirFilterLimit` | `100000` | Row threshold before `.filter/` value listing is restricted |
### Metadata TTLs
| Field | Default | Role |
|-------|---------|------|
| `MetadataRefreshInterval` | `10s` | Catalog cache (schemas, tables, views) |
| `StructuralMetadataRefreshInterval` | `5m` | PKs, permissions, row counts |
### NFS (macOS write path)
| Field | Default | Role |
|-------|---------|------|
| `NFSStreamingThreshold` | 10 MiB | Buffer size → streaming commit |
| `NFSMaxRandomWriteSize` | 100 MiB | Max random-write buffer |
| `NFSCacheReaperInterval` | `30s` | Stale entry check interval |
| `NFSCacheIdleTimeout` | `5m` | Idle force-commit |
### DDL, logging, identity, FUSE mode
| Field | Default | Role |
|-------|---------|------|
| `DDLGracePeriod` | `30s` | Completed DDL session visibility |
| `LogLevel` | `warn` | Zap level (`debug`, `info`, `warn`, `error`) |
| `LogFile` | empty | File path; empty → stderr |
| `LogFormat` | `text` | `text` or `json` |
| `LogSQLParams` | `false` | Log bind parameters (sensitive) |
| `UserID` | empty | Per-mount undo/log identity |
| `AutoSavepointInterval` | `30m` | Inactivity auto-savepoint (`0` off) |
| `UndoListLimit` | `100` | Default `.undo/` listing cap |
| `LegacyFuse` | `false` | Linux: legacy FUSE tree vs shared `fs.Operations` |
| `ConfigDir` | `GetDefaultConfigDir()` | Recorded config directory path |
## TLS enforcement
Applied in `db.enforceSSLMode` when opening the pool (`db.NewClient`), using `cfg.InsecureNoSSL`.
```text
enforceSSLMode(connStr, InsecureNoSSL)
|
+--------------------+--------------------+
| | |
localhost remote host insecure_no_ssl
(127.0.0.1, ::1, sslmode missing true → no change
unix socket) or disable/prefer + warn log
| |
no sslmode → → sslmode=require
sslmode=disable (require/verify-* kept)
explicit sslmode kept
```
When `true` (`--insecure-no-ssl`, `TIGERFS_INSECURE_NO_SSL`, or YAML), remote connections keep the connection string’s `sslmode` unchanged and log a warning. Default is enforced TLS for non-localhost hosts.
| Host class | No `sslmode` in URI | `sslmode=disable` / `prefer` | `require` / `verify-ca` / `verify-full` |
|------------|-------------------|------------------------------|----------------------------------------|
| Localhost / Unix socket | Adds `disable` | Unchanged | Unchanged (explicit override allowed) |
| Remote | Adds `require` | Replaced with `require` | Unchanged |
Localhost detection: `localhost`, `127.0.0.1`, `::1`, empty host, or host paths starting with `/`.
## Mount-specific overrides
Each `tigerfs mount` process loads config once, then patches the struct before `mountFilesystem`. These flags override file/env for that mount only:
| Flag | Config field | Behavior |
|------|--------------|----------|
| `--schema` | `DefaultSchema` | Set when non-empty |
| `--no-filename-extensions` | `NoFilenameExtensions` | Set `true` when passed |
| `--query-timeout` | `QueryTimeout` | Applied when `> 0` |
| `--dir-filter-limit` | `DirFilterLimit` | Applied when `> 0` |
| `--max-pipeline-depth` | `MaxPipelineDepth` | Applied only if flag **changed** (allows `0`) |
| `--legacy-fuse` | `LegacyFuse` | Linux legacy FUSE backend |
| `--insecure-no-ssl` | `InsecureNoSSL` | Disables TLS enforcement |
| `--user-id` | `UserID` | Flag → `TIGERFS_USER_ID` → value from config file |
| `--auto-savepoint-interval` | `AutoSavepointInterval` | Applied when `!= 0` (`>= 0` required) |
| `--undo-list-limit` | `UndoListLimit` | Applied when `> 0` |
`--max-ls-rows` is declared on the mount command (default `10000`) but is **not** wired to `DirListingLimit` in `mount.go`; use `dir_listing_limit` in config or `TIGERFS_DIR_LISTING_LIMIT` instead (default `1000`).
Root persistent flags (all subcommands):
| Flag | Viper key | Default |
|------|-----------|---------|
| `--config-dir` | `config_dir` | `GetDefaultConfigDir()` |
| `--log-level` | `log_level` | `warn` |
| `--log-file` | `log_file` | stderr |
| `--log-format` | `log_format` | `text` |
| `--log-sql-params` | `log_sql_params` | `false` |
`--debug` is not a separate config key; use `--log-level debug`.
### Per-mount identity
`UserID` tags undo/log entries and drives per-user filters on `.log/` and `.undo/.by/user_id/`. It is session-scoped (lost on remount). Precedence at mount: `--user-id` → `TIGERFS_USER_ID` → `user_id` from config file → anonymous.
## Validation rules
`tigerfs config validate` checks the on-disk file (syntax + unmarshal) and:
| Field | Rule |
|-------|------|
| `port` | 1–65535 |
| `pool_size` | ≥ 1 |
| `pool_max_idle` | ≥ 0 |
| `dir_listing_limit` | ≥ 1 |
| `default_format` | `tsv`, `csv`, or `json` |
| `binary_encoding` | `raw`, `base64`, or `hex` |
| `log_level` | `debug`, `info`, `warn`, or `error` |
## Inspecting configuration
Run after exporting env vars or editing the file:
```bash
tigerfs config show
```
Sensitive fields (`password`, Tiger secret keys) are omitted from output.
```bash
tigerfs config validate
```
```bash
export TIGERFS_QUERY_TIMEOUT=45s
tigerfs mount --schema public --user-id agent-7 \
postgres://user@db.example.com/mydb /mnt/db
```
`config show` reflects Viper merge (defaults + file + env + root flags). Mount-only flag patches apply only to that mount process after `Load()`.
## Related pages
Cobra commands, mount flags, and argument patterns beyond config keys.
`postgres://` URIs, `.pgpass`, pool behavior, and connection-string `sslmode` details.
`default_backend`, `tiger:` / `ghost:` prefixes, and Tiger Cloud credentials.
How `metadata_refresh_interval` and FUSE `attr_timeout` / `entry_timeout` affect freshness.
FUSE vs NFS: which config keys apply per platform.
TLS errors, stale metadata, and `--log-level debug`.
---
## 18. Data formats reference
> TSV, CSV, JSON, YAML row encoding, PATCH semantics, NULL representation, type extensions (.txt/.json/.bin), and binary_encoding options.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/18-data-formats-reference.md
- Generated: 2026-06-03T08:00:03.124Z
### Source Files
- `internal/tigerfs/format/json.go`
- `internal/tigerfs/format/csv.go`
- `internal/tigerfs/format/yaml.go`
- `internal/tigerfs/format/convert.go`
- `internal/tigerfs/fs/extensions.go`
- `test/integration/format_test.go`
---
title: "Data formats reference"
description: "TSV, CSV, JSON, YAML row encoding, PATCH semantics, NULL representation, type extensions (.txt/.json/.bin), and binary_encoding options."
---
TigerFS serializes PostgreSQL rows through the `format` package and exposes them as mount paths: row-as-file (`123.tsv`, `123.json`), row-as-directory column files (`email.txt`, `avatar.bin`), and pipeline exports under `.export/`. Reads always query the database; writes route through `fs.Operations` and `format.Parse*` helpers with either PATCH (format suffix) or PUT (bare path) semantics.
## Format overview
| Format | Row read path | Row write (PATCH) | Bulk export (default) | Bulk import |
|--------|---------------|-------------------|----------------------|-------------|
| TSV | `table/pk.tsv` or `table/pk/.tsv` | Header line + value line | No header row | Header row required (or `.no-headers/`) |
| CSV | `table/pk.csv` or `table/pk/.csv` | Header line + value line | No header row (RFC 4180 quoting) | Header row required (or `.no-headers/`) |
| JSON | `table/pk.json` or `table/pk/.json` | Object keys = columns | Pretty-printed array | JSON array of objects |
| YAML | `table/pk.yaml` or `table/pk/.yaml` | `key: value` document | Multi-document (`---` per row) | Multi-document YAML |
Default row serialization when no extension is given is TSV. Config key `formats.default_format` (`default_format` in `Config`) accepts `tsv`, `csv`, or `json` (default `tsv`); YAML is selected only by the `.yaml` path suffix.
```text
PostgreSQL row
│
▼
format.RowTo* / format.RowsTo* ← read (export, row file, column text)
│
▼
mount path bytes
mount write bytes
│
▼
format.Parse* / parseWriteData
│
▼
db.UpdateRow / InsertRow / Import*
```
## Row-as-file encoding
### TSV (default)
**Read:** One line, tab-separated values in **schema column order**, trailing newline, **no header row**. NULL → empty field (consecutive tabs for adjacent NULLs).
**Write (`.tsv` suffix, PATCH):** Exactly two lines after trimming trailing empty lines: line 1 = tab-separated column names, line 2 = tab-separated values. Only columns listed in the header are updated. More than two non-empty lines returns a parse error.
**Bare path (no extension, PUT):** Single line of tab-separated values in full schema column order (all columns). Used for paths like `table/123` without a format suffix; new row inserts require a format suffix on macOS NFS.
### CSV
Same layout as TSV but comma-delimited and **RFC 4180** quoting: fields containing `,`, `"`, or newlines are double-quoted; internal quotes are doubled.
**Read:** Headerless, schema order, NULL = empty field.
**Write (`.csv`, PATCH):** Header row + value row via `csv.Reader` (max two records).
### JSON
**Read:** Single compact object per row (`json.Encoder` with `SetEscapeHTML(false)`). NULL → JSON `null`. Booleans are JSON `true`/`false` (not PostgreSQL `t`/`f`). Numbers and strings use JSON-native types where possible (`NormalizeForJSON`).
**Multi-row export** (`.export/json`): Indented JSON array (`SetIndent("", " ")`), one object per row.
**Write (`.json`, PATCH):** Any JSON object; only keys present are updated. Omitted keys are left unchanged (not set to NULL). Explicit `null` sets NULL.
### YAML
**Read:** Document starts with `---`, then `column: value` lines per row. NULL → literal `null`. Scalars use `ConvertValueToText` then YAML quoting rules (reserved words like `null`, `true`, special characters, leading/trailing space).
**Multi-row export:** Each row is a separate document concatenated (each begins with `---`).
**Write (`.yaml`, PATCH):** Parsed as a single mapping; keys determine updated columns (same PATCH model as JSON).
## PATCH vs PUT write semantics
All paths with an explicit format extension (`.tsv`, `.csv`, `.json`, `.yaml`) use **PATCH**: only named columns change; others keep their stored values.
| Path pattern | Semantics | Parser |
|--------------|-----------|--------|
| `table/pk.json`, `table/pk.yaml` | PATCH | `ParseJSON`, `ParseYAML` |
| `table/pk.tsv`, `table/pk.csv` | PATCH | `ParseTSVWithHeader`, `ParseCSVWithHeader` |
| `table/pk` (no extension) | PUT — all columns in schema order | `ParseTSV` + schema column list |
Empty body with a format suffix is valid for insert paths (row created with PK from path and column defaults).
New rows via bare path without a format suffix are rejected (`newBarePathRejection`) to avoid NFS inode conflicts; use `.tsv`, `.json`, etc. PK columns from the path are merged into INSERT when omitted from the body.
**Example (PATCH JSON):**
```bash
echo '{"email":"new@example.com"}' > /mount/users/123.json
# UPDATE users SET email = ... WHERE id = 123; name unchanged
```
**Example (PATCH TSV):**
```bash
printf 'email\tname\nnew@example.com\tNew Name\n' > /mount/users/123.tsv
```
## NULL representation
| Surface | NULL on read | NULL on write | Notes |
|---------|--------------|---------------|-------|
| TSV / CSV row or export | Empty field | Empty field in value row | `,,` or `\t\t` between values |
| JSON row / export | `null` | `null` or omitted key | Omitted key on PATCH does **not** clear the column |
| YAML row / export | `null` | `null` | |
| Column file | 0-byte file | `rm column` sets NULL | Writing `""` stores empty string, not NULL |
| Bulk import | Empty field / JSON `null` | Same parsers as export | |
## Value conversion (`ConvertValueToText`)
Shared text rules for TSV, CSV, YAML scalars, and column files:
| PostgreSQL type | Text form |
|-----------------|-----------|
| `boolean` | `t` / `f` |
| `timestamp` / `time` | RFC3339Nano |
| `json` / `jsonb` | Compact JSON string |
| Arrays | JSON array string |
| `uuid` | Standard UUID string |
| `numeric` (pgtype) | JSON-marshaled decimal when available |
| `NULL` | Empty string |
JSON row encoding uses `NormalizeForJSON` instead, preserving JSON booleans and numeric types where compatible.
## Column filename extensions
Type-driven extensions map PostgreSQL types to suffixes; parameterized types strip the length suffix first (`varchar(255)` → `.txt`).
| PostgreSQL type | Extension | Example filename |
|-----------------|-----------|------------------|
| `text`, `varchar`, `char`, `bpchar` | `.txt` | `name.txt` |
| `json`, `jsonb` | `.json` | `metadata.json` |
| `xml` | `.xml` | `config.xml` |
| `bytea` | `.bin` | `avatar.bin` |
| `geometry`, `geography` | `.wkb` | `location.wkb` |
| Integer, numeric, boolean, date, arrays, etc. | (none) | `age`, `tags` |
`FindColumnByFilename` accepts either `column` or `column.ext` when the extension matches the column type. Disable extensions with `--no-filename-extensions` or `no_filename_extensions: true` (exact column names only).
Within a row directory, dot-prefixed format aliases expose the full row: `.json`, `.csv`, `.tsv`, `.yaml` (same encoding as top-level row files).
## `binary_encoding` configuration
BYTEA representation for filesystem I/O. Valid values: `raw`, `base64`, `hex`. Set in `~/.config/tigerfs/config.yaml` under `formats.binary_encoding`, via `TIGERFS_BINARY_ENCODING`, or validated at config load.
| Value | Intended use |
|-------|----------------|
| `raw` | BYTEA column files contain raw bytes (default; matches current column read/write) |
| `base64` | Config-valid; not wired in `format` or `fs.Operations` read/write paths today |
| `hex` | Config-valid; not wired in `format` or `fs.Operations` read/write paths today |
For binary column files today, read and write pass through byte content directly (`.bin` suffix is a naming hint). Use external tools (`base64`, `xxd`) to transform raw bytes when needed.
Synthesized file-first workspaces use a separate `encoding` column (`utf8` | `base64`) on backing tables in the `tigerfs` schema — not the mount-level `binary_encoding` setting.
## Bulk export and import
### Export (`.export/`)
`readExportFile` serializes pipeline-filtered rows:
| Path | Output |
|------|--------|
| `table/.export/json` | `RowsToJSON` — indented array |
| `table/.export/csv` | `RowsToCSV` — data rows only |
| `table/.export/.with-headers/csv` | `RowsToCSVWithHeaders` |
| `table/.export/tsv` (or default) | `RowsToTSV` or `RowsToTSVWithHeaders` |
| `table/.export/yaml` | `RowsToYAML` — multi-document |
Pipeline context (`.filter/`, `.order/`, `.first/`, `.columns/`, etc.) applies before serialization. Default row cap uses `dir_listing_limit` when no `.first`/`.last` limit is set.
### Import (`.import/`)
| Mode | Path segment | Behavior |
|------|--------------|----------|
| Append | `.import/.append/` | Insert only; PK conflict fails |
| Sync | `.import/.sync/` | Upsert by primary key |
| Overwrite | `.import/.overwrite/` | Delete all rows, then insert |
Parsers: `ParseCSVBulk`, `ParseTSVBulk`, `ParseJSONBulk`, `ParseYAMLBulk` (header/document rules as above). `.import/.../.no-headers/csv` or `.no-headers/tsv` uses schema column order via `ParseCSVBulkNoHeaders` / `ParseTSVBulkNoHeaders`. Default import format when unspecified is `csv`.
## Primary key and path caveats
Text primary keys that end with `.json`, `.csv`, `.tsv`, or `.yaml` are parsed as format extensions (e.g. PK `config` + format `json` for path `config.json`). Integer and UUID keys are unaffected.
Row files (`1.json`) are reachable by direct path but omitted from `ls` at the table level to keep listings small; row directories (`1/`) appear in listings.
## Configuration quick reference
| Key | Env var | Default | Applies to |
|-----|---------|---------|------------|
| `formats.default_format` | `TIGERFS_DEFAULT_FORMAT` | `tsv` | Default row format preference (`tsv`, `csv`, `json`) |
| `formats.binary_encoding` | `TIGERFS_BINARY_ENCODING` | `raw` | BYTEA (`raw`, `base64`, `hex` — see above) |
| `filesystem.no_filename_extensions` | `TIGERFS_NO_FILENAME_EXTENSIONS` | `false` | Column extension mapping |
| `filesystem.trailing_newlines` | `TIGERFS_TRAILING_NEWLINES` | `true` | Column and `.info/count` reads |
## Related pages
Row and column reads, PATCH writes, import/export paths, and pipeline pagination.
`.export/`, `.import/`, `.columns/`, and pipeline chaining that shapes bulk format output.
Mount hierarchy, row-as-file vs row-as-directory, and path-to-SQL mapping.
Full `Config` fields, precedence, and environment variable names.
---
## 19. Connection reference
> postgres:// URIs, PG* env vars, password_command, .pgpass, sslmode=require rules, localhost exemptions, and pool settings.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/19-connection-reference.md
- Generated: 2026-06-03T08:02:25.722Z
### Source Files
- `internal/tigerfs/db/connection.go`
- `internal/tigerfs/db/password.go`
- `internal/tigerfs/db/tls.go`
- `internal/tigerfs/cmd/test_connection.go`
- `docs/spec.md`
---
title: "Connection reference"
description: "postgres:// URIs, PG* env vars, password_command, .pgpass, sslmode=require rules, localhost exemptions, and pool settings."
---
TigerFS opens PostgreSQL through `db.NewClient`, which resolves credentials, enforces TLS on connection strings, and builds a `pgxpool` pool used by every mount backend. Connection strings come from CLI arguments, cloud backend CLIs (`tiger:`, `ghost:`), or standard `PG*` / `TIGERFS_*` configuration; `tigerfs test-connection` additionally uses `db.ResolveConnectionString` to assemble a URL from config when no argument is passed.
## Connection string formats
TigerFS accepts the same connection string shapes as [pgx](https://github.com/jackc/pgx): URL form and libpq key-value form.
| Format | Example | Notes |
|--------|---------|-------|
| URL | `postgres://user@host:5432/dbname` | Preferred for mounts; supports query params (`?sslmode=require`) |
| URL (alternate scheme) | `postgresql://user@host/db` | Treated identically to `postgres://` |
| Key-value | `host=db.example.com user=myuser dbname=mydb` | Parsed for TLS host extraction and `sslmode` injection |
URL structure:
```text
postgres://[user[:password]@]host[:port][/dbname][?param=value&...]
```
Passwords embedded in URLs appear in process listings and logs. `db.SanitizeConnectionString` strips URL passwords and masks `password=` in key-value strings before logging; prefer `user@host` URLs plus a separate secret source.
## How TigerFS picks a connection target
Resolution differs by command.
### Mount (`tigerfs mount`)
| Input | `connStr` passed to `NewClient` | Mountpoint |
|-------|----------------------------------|------------|
| `tigerfs mount postgres://user@host/db /mnt` | `postgres://user@host/db` | `/mnt` |
| `tigerfs mount tiger:ID` | Resolved `postgres://…` from Tiger CLI | `/ID` |
| `tigerfs mount /mnt` (no URL) | `""` (empty; pgx uses `PG*` env vars) | `/mnt` |
Cloud prefixes are resolved in `cmd/mount.go` via `backend.Resolve` → `GetConnectionString` before connect.
### `test-connection` and `migrate`
These commands call `db.ResolveConnectionString` when no explicit argument is given:
1. Explicit connection argument (if any)
2. `TIGER_SERVICE_ID` / `connection.tiger_service_id` → Tiger Cloud CLI
3. Config `host` (from `PGHOST` / config file) → builds `postgres://{user}@{host}:{port}/{database}`
If none apply, resolution fails with an error asking for a connection string, Tiger service ID, or `PGHOST`.
```mermaid
sequenceDiagram
participant CLI as tigerfs CLI
participant Resolve as db.ResolveConnectionString
participant NC as db.NewClient
participant Pool as pgxpool
CLI->>Resolve: explicit / config / Tiger Cloud
Resolve-->>CLI: postgres URL or error
CLI->>NC: connStr + Config
NC->>NC: resolvePassword
NC->>NC: enforceSSLMode
NC->>Pool: ParseConfig, Ping
Pool-->>NC: ready
```
For remote hosts, pass an explicit `postgres://user@remote-host:5432/db` on **mount** so `enforceSSLMode` can read the hostname from the URL. An empty mount connection string is treated as localhost for TLS defaults (see [TLS enforcement](#tls-enforcement)); pgx may still connect using `PGHOST` from the environment.
## PostgreSQL environment variables
Viper binds standard libpq variables into `config.Config` at startup (`config.Init`):
| Variable | Config field | Default when unset |
|----------|--------------|-------------------|
| `PGHOST` | `host` | (empty) |
| `PGPORT` | `port` | `5432` |
| `PGUSER` | `user` | (empty) |
| `PGDATABASE` | `database` | (empty) |
| `PGPASSWORD` | `password` | (empty; not used directly by `NewClient`) |
TigerFS-prefixed overrides use `TIGERFS_` + map key (e.g. `TIGERFS_PORT` overrides `PGPORT` when both are set).
Additional Tiger connection env vars:
| Variable | Purpose |
|----------|---------|
| `TIGERFS_PASSWORD` | Database password (after `PGPASSWORD` in precedence) |
| `TIGERFS_PASSWORD_COMMAND` | Shell command whose stdout is the password |
| `TIGERFS_POOL_SIZE` | Maps to `pool_size` (default `10`) |
| `TIGERFS_POOL_MAX_IDLE` | Maps to `pool_max_idle` (default `5`) |
| `TIGERFS_INSECURE_NO_SSL` | Skips remote TLS enforcement when `true` |
| `TIGER_SERVICE_ID` | Legacy Tiger Cloud service fallback in `ResolveConnectionString` |
The `password` field exists on `Config` for display and viper binding, but `db.resolvePassword` reads `PGPASSWORD` / `TIGERFS_PASSWORD` from the environment only—not `cfg.Password`. Do not put passwords in `config.yaml`; use `.pgpass`, env vars, or `password_command`.
## Password resolution
`db.NewClient` resolves secrets before `pgxpool.ParseConfig`:
```mermaid
flowchart TD
A[Connection string already contains password?] -->|yes| B[pgx uses URL password as-is]
A -->|no| C{PGPASSWORD set?}
C -->|yes| D[Inject into URL / append key-value]
C -->|no| E{TIGERFS_PASSWORD set?}
E -->|yes| D
E -->|no| F{password_command configured?}
F -->|yes| G[Run command, inject stdout]
F -->|no| H[pgx reads ~/.pgpass]
```
| Priority | Source | Behavior |
|----------|--------|----------|
| 1 (implicit) | Password in `postgres://user:pass@…` | Left unchanged; env/command not consulted |
| 2 | `PGPASSWORD` | Injected into URL; wins over `TIGERFS_PASSWORD` and `password_command` |
| 3 | `TIGERFS_PASSWORD` | Injected when `PGPASSWORD` unset |
| 4 | `password_command` | Executed when env passwords unset |
| 5 | `~/.pgpass` | Handled by pgx when no injected password |
### `password_command`
Configure in `~/.config/tigerfs/config.yaml`:
```yaml
connection:
password_command: "vault kv get -field=password secret/prod-db"
```
Or via `TIGERFS_PASSWORD_COMMAND`.
Shell command run to fetch the database password. First token is the executable; remaining tokens are arguments (`strings.Fields`—no quoted-argument parsing). Stdout is trimmed; must be non-empty. Context timeout: **5 seconds**. Non-zero exit includes stderr in the error.
```bash
export TIGERFS_PASSWORD_COMMAND='pass show tigerfs/prod'
tigerfs test-connection postgres://app@db.example.com/prod
```
### `.pgpass`
TigerFS does not parse `.pgpass` itself. When no password is injected, [pgx](https://github.com/jackc/pgx) reads `~/.pgpass` using the usual libpq rules.
```text
hostname:port:database:username:password
```
```bash
chmod 0600 ~/.pgpass
tigerfs mount postgres://myuser@localhost:5432/mydb /mnt/db
```
### Password injection rules
- **URL:** Requires `user@host`; replaces or adds `user:password@`.
- **Key-value:** Appends `password=…` only if `password=` is absent; existing key-value passwords are not overwritten.
If `PGPASSWORD` is set but the connection string has no `user@host` segment, injection fails with `connection string has no user@host format`.
## TLS enforcement
`db.enforceSSLMode` runs on every `NewClient` call after password handling.
### Remote hosts (non-localhost)
| Current `sslmode` | Action |
|-------------------|--------|
| (missing) | Add `sslmode=require` |
| `disable`, `prefer` | Replace with `sslmode=require` |
| `require`, `verify-ca`, `verify-full` | Unchanged |
### Localhost exemption
Hosts treated as local (no forced `require`):
- `localhost`, `127.0.0.1`, `::1`, empty host
- Unix socket paths (host starts with `/`)
When no `sslmode` is present on a local connection, TigerFS adds **`sslmode=disable`** (avoids pgx `prefer` handshake issues on `ssl=off` clusters). An explicit `sslmode` in the connection string is always preserved on localhost (e.g. `sslmode=require` on `localhost` stays as written).
### Disabling enforcement
CLI: `--insecure-no-ssl` on `mount` and `migrate`. Config: `connection.insecure_no_ssl` or `TIGERFS_INSECURE_NO_SSL=true`. Skips TLS rewriting; logs a warning for remote hosts.
`--insecure-no-ssl` allows plaintext connections to remote databases. Use only for local development or trusted networks.
## Connection pool settings
Each mount daemon holds one `pgxpool.Pool` for the lifetime of the process.
| Setting | Config key | Env var | Default | pgx mapping |
|---------|------------|---------|---------|-------------|
| Max connections | `connection.pool_size` | `TIGERFS_POOL_SIZE` | `10` | `MaxConns` |
| Min idle connections | `connection.pool_max_idle` | `TIGERFS_POOL_MAX_IDLE` | `5` | `MinConns` |
Validation (`tigerfs config` write path): `pool_size` ≥ 1; `pool_max_idle` ≥ 0.
Additional pool behavior in `db.NewClient`:
- **Ping:** Caller context must succeed before the client is returned.
- **Pool creation:** Uses `context.Background()` so short mount timeouts do not cancel background pool maintenance.
- **`AfterConnect`:** Sets PostgreSQL `statement_timeout` from `query_timeout` when configured (default 30s in config).
- **Tracing:** Optional SQL debug tracing via `log_sql_params`.
- **Shutdown:** `Client.Close()` closes the pool on unmount.
Concurrent filesystem operations share the pool; server-side poolers (PgBouncer, etc.) work transparently.
Size `pool_size` to your database `max_connections` budget across all TigerFS mounts on the same instance. Each mount process opens up to `pool_size` connections.
## Verify connectivity
```bash
tigerfs test-connection postgres://user@localhost:5432/mydb
```
With only environment configuration:
```bash
export PGHOST=localhost PGDATABASE=mydb PGUSER=myuser
tigerfs test-connection
```
Successful output includes PostgreSQL version, current database/user, and counts of accessible schemas and tables (system/internal schemas excluded).
On failure, check stderr for structured zap hints (TLS, `password_command`, parse errors) before errno from mount tools.
## Security practices
| Practice | Recommendation |
|----------|----------------|
| Production secrets | `~/.pgpass` or `password_command` |
| Development / CI | `PGPASSWORD` or `TIGERFS_PASSWORD` |
| Remote encryption | Explicit `postgres://…` URL; avoid relying on empty-string mount + `PGHOST` for TLS defaults |
| Config file | Never store `password:` in `config.yaml` |
| Logging | Passwords stripped via `SanitizeConnectionString` in registry and debug logs |
## Related pages
Full config file layout, precedence, and TigerFS-specific env vars.
tiger: and ghost: prefix resolution and CLI credential requirements.
mount, test-connection, migrate flags and argument patterns.
Connection failures, TLS errors, and errno recovery.
---
## 20. Error codes
> fs.ErrorCode to POSIX errno mapping, structured zap hints on failure, constraint violations, and reading stderr when tools report EINVAL/EACCES.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/20-error-codes.md
- Generated: 2026-06-03T08:01:19.438Z
### Source Files
- `internal/tigerfs/fs/errors.go`
- `internal/tigerfs/fs/errors_test.go`
- `internal/tigerfs/fuse/control_files.go`
- `internal/tigerfs/logging/logging.go`
- `docs/spec.md`
---
title: "Error codes"
description: "fs.ErrorCode to POSIX errno mapping, structured zap hints on failure, constraint violations, and reading stderr when tools report EINVAL/EACCES."
---
TigerFS surfaces filesystem failures as POSIX `errno` values to shells and tools (`mv`, `cp`, `echo > file`), while the shared `fs` package classifies errors with backend-agnostic `ErrorCode` values wrapped in `*FSError`. FUSE and NFS adapters translate `FSError` to `syscall.Errno` or `os` errors; structured detail—including optional `Hint` fields—is written to **stderr** via zap before the syscall returns.
## Error flow
FUSE and NFS cannot return arbitrary error strings to callers. The design logs human-oriented detail first, then returns a standard errno.
```mermaid
sequenceDiagram
participant Tool as Shell / tool
participant Adapter as fuse.FSAdapter or nfs.OpsFilesystem
participant Ops as fs.Operations
participant DB as PostgreSQL
Tool->>Adapter: VFS op (write, rename, stat)
Adapter->>Ops: ReadDir / Stat / WriteFile / ...
Ops->>DB: SQL (when needed)
DB-->>Ops: success or driver error
Ops-->>Adapter: nil or *FSError
Note over Adapter: logging.Error / Debug
with hint, message, cause
Adapter-->>Tool: errno (e.g. EINVAL, EACCES)
```
On Linux (FUSE), `FSAdapter.ErrorToErrno` is the canonical mapping used by migrated `OpsNode` paths and adapter helpers. macOS (NFS) maps a subset of codes to `os.ErrNotExist`, `os.ErrPermission`, and `os.ErrInvalid`; other `FSError` codes may collapse differently at the NFS layer.
## `ErrorCode` and `FSError`
`ErrorCode` is an `int` enum defined in `internal/tigerfs/fs/errors.go`. Each constant documents its intended POSIX mapping in comments; adapters implement that mapping in `fuse/adapter.go`.
| `ErrorCode` | Constant | POSIX errno | Typical causes |
|-------------|----------|-------------|----------------|
| 0 | `ErrNone` | success (`0`) | No error |
| 1 | `ErrNotExist` | `ENOENT` | Missing path, row, column, or DDL session |
| 2 | `ErrPermission` | `EACCES` | Read-only view, reserved TigerFS name, immutable `.history/`, undo across migration boundary, DB privilege denial |
| 3 | `ErrInvalidPath` | `EINVAL` | Malformed pipeline path, wrong capability order, invalid DDL path |
| 4 | `ErrInvalidFormat` | `EINVAL` | Unparseable row file (JSON/TSV/CSV/YAML) |
| 5 | `ErrInvalidOperation` | `EPERM` | Operation not allowed (e.g. `Readlink` on non-symlink) |
| 6 | `ErrReadOnly` | `EROFS` | Write on read-only mount |
| 7 | `ErrNotEmpty` | `ENOTEMPTY` | `rmdir` on non-empty directory |
| 8 | `ErrAlreadyExists` | `EEXIST` | Path or row already exists (`ErrExists` alias) |
| 9 | `ErrIO` | `EIO` | DB connection/query failure, unexpected SQL error, many wrapped `Cause` errors |
| 10 | `ErrInternal` | `EIO` | Unexpected internal failure |
| 11 | `ErrNotImplemented` | `ENOSYS` | Feature not implemented |
| 12 | `ErrInvalidArgument` | `EINVAL` | Invalid argument value (grouped with path/format errors in FUSE) |
`FSError` carries programmatic and user-facing fields:
| Field | Role |
|-------|------|
| `Code` | `ErrorCode` for adapter mapping |
| `Message` | Short description; also `error.Error()` when no `Cause` |
| `Cause` | Underlying error (`Unwrap` for `errors.Is` / `As`) |
| `Hint` | Actionable guidance logged to stderr before errno return |
Constructors such as `NewNotExistError`, `NewPermissionError`, and `NewInvalidPathError` populate `Message`; `NewInvalidPathError` also copies `reason` into `Hint`.
`ErrNotExist` is treated as a normal outcome for many lookups (tab completion, `stat` before `mkdir`). `ErrorToErrno` logs it at **debug** only, not `Error`, to avoid noise.
## FUSE mapping (`ErrorToErrno`)
`fuse.FSAdapter.ErrorToErrno` converts `*FSError` to `syscall.Errno` and applies logging rules:
| Condition | Log level | Fields |
|-----------|-----------|--------|
| `Code == ErrNotExist` | `logging.Debug` | `message`, optional `hint`, `cause` |
| `Hint != ""` (other codes) | `logging.Error` | `message`, `hint`, `cause` |
| `Cause != nil`, no hint | `logging.Debug` | `message`, `cause` |
| Otherwise | (no log) | — |
Mapping switch (default unknown codes → `EIO`):
```
ErrNotExist → ENOENT
ErrPermission → EACCES
ErrInvalidPath,
ErrInvalidFormat,
ErrInvalidArgument → EINVAL
ErrInvalidOperation → EPERM
ErrReadOnly → EROFS
ErrNotEmpty → ENOTEMPTY
ErrAlreadyExists → EEXIST
ErrIO, ErrInternal → EIO
ErrNotImplemented → ENOSYS
(default) → EIO
```
Unit tests in `fuse/adapter_test.go` lock these mappings for the covered codes.
## NFS mapping differences
`nfs.OpsFilesystem` does not call `ErrorToErrno`. For `Stat`, `ReadDir`, and `Rename` it maps:
| `ErrorCode` | Go error returned |
|-------------|-------------------|
| `ErrNotExist` | `os.ErrNotExist` |
| `ErrPermission` | `os.ErrPermission` |
| `ErrInvalidPath`, `ErrInvalidArgument` | `os.ErrInvalid` |
| Other (including `ErrIO`, `ErrInvalidFormat`) | Often `os.ErrNotExist` on directory ops, or wrapped `fmt.Errorf` on writes |
Create/Open paths log `message`, `hint`, and `cause` at `Error` and return `os.ErrInvalid` when `ValidateCreate` rejects a path (e.g. bare row path without format suffix on macOS).
Do not assume NFS and FUSE return identical errno for every `FSError`. Prefer stderr logs and `Hint` when diagnosing macOS mounts.
## PostgreSQL and constraint violations
The specification (`docs/spec.md`) documents the user-visible contract: standard errno at the syscall boundary, full PostgreSQL detail in logs.
| PostgreSQL / domain situation | Spec errno | `ErrorCode` in shared `fs` layer (when used) |
|------------------------------|------------|-----------------------------------------------|
| Row not found | `ENOENT` | `ErrNotExist` |
| Table/view privilege denied | `EACCES` | `ErrPermission` |
| NOT NULL / FK / check violation | `EACCES` (generic message to tool) | Often `ErrIO` with `Cause` = driver error on `WriteFile`/`InsertRow`; legacy FUSE nodes may map substring `"constraint"` → `EACCES` |
| Connection / timeout / other SQL | `EIO` | `ErrIO` + `Cause` |
**Pre-insert validation:** `db.ValidateConstraints` checks NOT NULL (no default) and UNIQUE before some incremental row commits (`fuse/partial.go`). Violations return Go errors with messages like `NOT NULL constraint violation on column email`; these are logged at `Error` in partial-row commit paths.
**Legacy FUSE paths** (still present alongside `fs.Operations`):
- `TableNode` delete/rmdir: errors containing `foreign key` or `constraint` → `EACCES`
- `RowDirectoryNode` unlink on NOT NULL column without default → `EACCES`
- `RowFileNode` flush failures → `EIO` (constraint text only in logs)
**Shared `fs.Operations` path:** many database failures become `ErrIO` with the driver error in `Cause` (e.g. `failed to insert row`). Tools see `EIO`; read stderr for `null value in column ... violates not-null constraint` and similar.
Example spec workflow: `echo 'Name' > /mnt/db/users/125/name` when `email` is NOT NULL may return exit code **13** (`EACCES`) on some code paths, while the mount process logs the real constraint message on stderr.
## `Hint` field patterns
`Hint` is set at the point of failure so adapters can log it before returning errno. Common sources:
| Area | Example hint |
|------|----------------|
| Path parsing (`fs/path.go`) | `filters must come before .order/ in the path` |
| Row create validation | `retry as "1.json", "1.tsv", or "1.csv" (bare-path writes conflict ...)` |
| DDL staging | `run validation with: touch /.create//.test` |
| Synth reserved names | `choose a different filename to avoid collision with TigerFS control directories` |
| History / undo | `history files are archived versions and cannot be modified`; migration boundary text from metadata `Description` |
| File-first routing | `use mount/.tables//... for data-first access to the backing table` |
Reserved TigerFS capability names (`.filter`, `.history`, `.undo`, etc.) return `ErrPermission` → `EACCES`, matching the spec reserved-name rule.
## Logging and reading stderr
The `tigerfs` CLI initializes zap in `logging.Init` from `--log-level` (`debug`, `info`, `warn`, `error`; default `warn`). Logs go to **stderr** unless `--log-file` is set. `--log-format json` emits structured lines; production-style configs use a compact JSON shape with a `message` key.
Run the mount in the foreground with debug logging:
```bash
tigerfs mount --foreground --log-level debug postgres://host/db /mnt/db
```
```bash
mv 1-foo-o.md a-foo-o.md
# or: echo 'x' > /mnt/db/users/125/name
echo $? # 13 = EACCES, 22 = EINVAL
```
Look for JSON or text lines immediately before the tool’s errno message:
```json
{"message":"invalid path ...","hint":"limit must be a positive integer"}
```
When `Hint` is set, `ErrorToErrno` logs at `Error` with `hint` as a zap field. The shell only prints the generic errno string (e.g. `Invalid argument`, `Permission denied`).
| Tool exit / errno | Value | Often means |
|-------------------|-------|-------------|
| `EINVAL` | 22 | Bad path grammar, bad format, invalid create path, NFS `ValidateCreate` rejection |
| `EACCES` | 13 | Permission, read-only view, reserved name, constraint (legacy paths), undo boundary |
| `EPERM` | 1 | `ErrInvalidOperation` (wrong op type) |
| `ENOENT` | 2 | Missing file/row/column |
| `EIO` | 5 | DB/network failure; check `Cause` in logs |
| `ENOSYS` | 38 | Not implemented |
FUSE returns errno only—never SQL text. Always correlate the user command with the **tigerfs** process stderr from the same time window, not only the shell’s one-line `mv: ...` message.
## errno quick reference for agents
When automating against a mount:
1. Treat non-zero exit codes as `errno`, not as PostgreSQL error codes.
2. On `EINVAL` or `EACCES`, search mount stderr for `"hint"` or `"message"` (JSON) or the plain message (text format).
3. Prefer fixing path grammar and capability order (see capability directories docs) before retrying writes.
4. On `EIO`, inspect `Cause` in logs for connection, timeout, or constraint strings from the driver.
5. Use `ErrNotExist` outcomes for existence checks without treating them as mount failures.
## Related pages
Path types, dot-directories, and how operations become SQL.
Pipeline grammar errors that surface as EINVAL with path hints.
`.test` / `.commit` validation errno and staging control files.
Undo boundary refusals (ErrPermission / EACCES) and migration metadata.
Stale mounts, `--log-level debug`, and common errno recovery.
`adapter_test.go`, integration tests, and error regression coverage.
---
## 21. Demo environment
> scripts/demo workflows (Docker vs macOS native), seed data, demo.sh start/shell/stop, and sample exploration commands.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/21-demo-environment.md
- Generated: 2026-06-03T08:01:27.061Z
### Source Files
- `scripts/demo/demo.sh`
- `scripts/demo/README.md`
- `scripts/demo/init.sql`
- `scripts/demo/seed.sh`
- `test/integration/demo_data.go`
---
title: "Demo environment"
description: "scripts/demo workflows (Docker vs macOS native), seed data, demo.sh start/shell/stop, and sample exploration commands."
---
The `scripts/demo/` tree provides a self-contained TigerFS playground: PostgreSQL loads relational seed data from `init.sql`, `demo.sh` mounts the database, and `seed.sh` creates synthesized file-first apps through the real `.build/` mount path.
## Layout
```text
scripts/demo/
demo.sh # start | stop | status | restart | shell
init.sql # data-first tables (PostgreSQL init)
seed.sh # file-first apps (via mount)
README.md
docker/
docker-compose.yml # postgres + tigerfs (FUSE)
Dockerfile # TigerFS image + tooling
mac/
docker-compose.yml # postgres only (TigerFS on host)
```
## Runtime modes
`demo.sh` picks a mode from flags or the host OS. Darwin defaults to macOS native NFS; other platforms default to Docker FUSE.
| Mode | Flag | PostgreSQL | TigerFS | Mount path | Connection string |
|------|------|------------|---------|------------|-------------------|
| Docker | `--docker` (default on Linux) | Container | Container (FUSE) | `/mnt/db` (inside `tigerfs`) | `postgres://demo:demo@postgres:5432/demo` |
| macOS native | `--mac` (default on macOS) | Container on `localhost:5432` | Host binary (`bin/tigerfs`) | `/tmp/tigerfs-demo` | `postgres://demo:demo@localhost:5432/demo` |
Both modes mount with `tigerfs mount --insecure-no-ssl` (demo-only TLS bypass for localhost/Docker). Pass `--debug` on any command to add `--log-level debug` to the mount.
```mermaid
sequenceDiagram
participant User
participant demo_sh as demo.sh
participant Compose as docker compose
participant PG as postgres (timescaledb-ha:pg18)
participant TFS as TigerFS
participant Seed as seed.sh
User->>demo_sh: start
demo_sh->>Compose: up -d
Compose->>PG: init.sql on first boot
demo_sh->>PG: wait pg_isready
demo_sh->>TFS: mount CONN_STR MOUNTPOINT
demo_sh->>Seed: seed.sh MOUNTPOINT
Seed->>TFS: .build/* + file writes
```
## Commands
Run all commands from `scripts/demo/` (repository-relative paths below assume that cwd).
```bash
./demo.sh start
# or: ./demo.sh start --docker
# or: ./demo.sh start --mac
```
Starts PostgreSQL, waits for init (`PostgreSQL init process complete` in logs, then `pg_isready`), mounts TigerFS, and runs `seed.sh`. Exits with an error if the mount is already active.
On macOS, use the host mount directly:
```bash
ls /tmp/tigerfs-demo/
cat /tmp/tigerfs-demo/users/1.json
```
In Docker mode, enter the container at the mount root:
```bash
./demo.sh shell
# working directory: /mnt/db
ls
cat users/1.json
```
```bash
./demo.sh stop
```
Docker: `umount` inside the container, then `docker compose down`. macOS: `pkill` tigerfs for the mountpoint, `umount` / `diskutil unmount force` on `/tmp/tigerfs-demo`, then stop the postgres container.
| Command | Behavior |
|---------|----------|
| `start` | Compose up, wait for DB, mount, seed file-first apps |
| `stop` | Unmount TigerFS, tear down containers / processes |
| `status` | Report container and mount state |
| `restart` | `stop`, pause, `start` |
| `shell` | Interactive shell at mount root (`/mnt/db` or `/tmp/tigerfs-demo`) |
`start` refuses to run if TigerFS is already mounted at the mode’s mountpoint. Run `./demo.sh stop` first.
## Seed data
Demo content splits into two phases: SQL at database init, filesystem writes after mount.
### Data-first (`init.sql`)
Loaded by PostgreSQL via `docker-entrypoint-initdb.d/init.sql` on first container start (`timescale/timescaledb-ha:pg18`, user/password/database `demo`).
| Object | Rows / notes |
|--------|----------------|
| `users` | 1,000 — `SERIAL` PK, composite index on `(last_name, first_name)` |
| `categories` | 10 — `TEXT` slug PK |
| `products` | 200 — `SERIAL` PK, FK to `categories` |
| `orders` | 8,000 — `UUID` PK with `uuidv7()` default |
| `product_inventory` | ~600 — composite PK `(product_id, warehouse)` |
| `product_views` | ~50,000 — TimescaleDB hypertable with columnstore |
| `active_users` | Updatable view on `users` |
| `order_summary` | JOIN view (orders + users + products + categories) |
Also creates schema `tigerfs`, enables TimescaleDB, and indexes for `.by/` navigation (`idx_products_category`, `idx_orders_*`, `idx_users_name`, `idx_orders_status_date`, etc.).
File-first apps are intentionally **not** in `init.sql`; comments in that file point to `seed.sh`.
### File-first (`seed.sh`)
Argument: mount path (`seed.sh `). Invoked by `demo.sh` after a successful mount.
| App | `.build/` spec | Content |
|-----|----------------|---------|
| `blog/` | `markdown,history` | 5 posts in `tutorials/`, `deep-dives/`; savepoints; history via rewrites |
| `docs/` | `markdown,history` | 4 pages in `getting-started/`, `reference/` |
| `snippets/` | `txt,history` | 3 plain-text files in `meetings/` |
Each app uses `echo 'format,history' > "$MOUNT/.build/"` then `mkdir` and `cat` heredocs into `.md` or `.txt` files. Blog and docs pages are written twice to populate `.history/`.
## Sample exploration
### Data-first tables
```bash
# Row read
cat users/1.json
# Pipeline / export
cat products/.by/category/electronics/.export/json
# Sampling
ls orders/.sample/10/
# Composite index path (users)
ls users/.by/last_name.first_name/Smith/
# Views
cat active_users/.export/json
ls order_summary/.first/5/
```
### File-first apps
```bash
ls blog/
cat blog/hello-world.md
cat docs/getting-started/installation.md
cat snippets/todo.txt
# History (blog and docs)
ls docs/.history/getting-started/installation.md/
ls blog/.savepoint/
```
On macOS after `./demo.sh start --mac`, prefix paths with `/tmp/tigerfs-demo/`. In Docker `./demo.sh shell`, paths are relative to `/mnt/db`.
## Docker image details
The `tigerfs` service image (`scripts/demo/docker/Dockerfile`) builds `cmd/tigerfs` on Go 1.25, installs FUSE3, `psql`, and optional agent tooling: Node.js, `@anthropic-ai/claude-code`, `tiger-cli`, and `ghost`. The container runs **privileged** for FUSE.
To use Claude Code inside the Docker demo, set `ANTHROPIC_API_KEY` before `start`, then `./demo.sh shell` and run `claude`.
Compose wires `ANTHROPIC_API_KEY` from the host environment into the `tigerfs` service.
## Prerequisites
| Mode | Requirements |
|------|----------------|
| Docker | Docker with Compose v2, daemon running, privileged FUSE in container |
| macOS | macOS, Docker Desktop, Go 1.21+ (builds `bin/tigerfs` if missing) |
## Integration tests vs demo dataset
`test/integration/demo_data.go` seeds a **smaller, deterministic** dataset for automated tests (`DefaultDemoConfig`: 100 users, 20 products, 200 orders, fixed `BaseTimestamp`). It mirrors the same table shapes and `.by/` indexes but is not invoked by `demo.sh`. Full demo scale and `uuidv7()` orders live only in `init.sql`.
Use the demo environment for manual exploration; use `seedDemoData` in integration tests for CI-stable assertions.
## Troubleshooting
| Symptom | Mitigation |
|---------|------------|
| Port `5432` in use (macOS) | Stop local PostgreSQL (`brew services stop postgresql`), inspect with `lsof -i :5432` |
| Mount stuck at `/tmp/tigerfs-demo` | `./demo.sh status --mac`; `diskutil unmount force /tmp/tigerfs-demo`; then `./demo.sh stop` |
| Docker mount failed | Check `docker compose -f scripts/demo/docker/docker-compose.yml ps` and container logs |
| `Go is not installed` (macOS) | Install Go 1.21+ or pre-build `bin/tigerfs` at repo root |
PostgreSQL init runs only on a **fresh** volume. To reload `init.sql`, remove the compose volume (`docker compose down -v`) before `start`.
## Verification signals
After `./demo.sh start`:
- **macOS:** `mount | grep /tmp/tigerfs-demo` shows the NFS mount; `ls /tmp/tigerfs-demo` lists `users`, `products`, `categories`, `orders`, `blog`, `docs`, `snippets`.
- **Docker:** `./demo.sh status` reports containers `RUNNING` and mount `MOUNTED` at `/mnt/db`; `docker compose exec tigerfs ls /mnt/db` lists the same top-level dirs.
`seed.sh` ends with a summary of created apps; missing directories usually mean mount or seed failed earlier in `start`.
## Related pages
First mount and explore workflows that reference `demo.sh` on Docker and macOS.
Platform prerequisites (FUSE, NFS, Docker) before running the demo.
How `blog/`, `docs/`, and `snippets/` map to backing tables and `.build/` specs.
Row reads, `.by/`, `.export/`, `.sample/`, and pipeline paths on seeded tables.
`.history/`, `.savepoint/`, and version browsing on seeded markdown apps.
`.build/` format strings (`markdown,history`, `txt,history`) used by `seed.sh`.
---
## 22. Workflow recipes
> Copy-paste file-first patterns: kanban (todo/doing/done + mv), knowledge base, session context, from skills/tigerfs/recipes.md.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/22-workflow-recipes.md
- Generated: 2026-06-03T08:01:15.039Z
### Source Files
- `skills/tigerfs/recipes.md`
- `skills/tigerfs/files.md`
- `docs/file-first.md`
- `README.md`
- `test/integration/fs_operations_test.go`
---
title: "Workflow recipes"
description: "Copy-paste file-first patterns: kanban (todo/doing/done + mv), knowledge base, session context, from skills/tigerfs/recipes.md."
---
TigerFS file-first workflows are defined in `skills/tigerfs/recipes.md` and implemented as synthesized markdown workspaces: write a format string to `mount/.build/`, use normal file tools (`Write`, `Glob`, `Grep`, `mv`, `rm`, `mkdir`), and optionally enable `history` for `.history/`, `.log/`, `.savepoint/`, and `.undo/`. Kanban and similar apps treat **directories as state** and **`mv` as transitions** (updating the backing `filename` column); savepoint/undo recipes use the same dot-directories documented in the file-first skill pack.
## Prerequisites
| Requirement | Detail |
|-------------|--------|
| Mount | `tigerfs mount CONNECTION MOUNTPOINT` (see CLI reference) |
| Workspace creation | `echo '' > mount/.build/` |
| History (optional) | Append `,history` to the format (e.g. `markdown,history`) |
| Multi-agent identity | `--user-id ` or `TIGERFS_USER_ID` at mount; runtime: `echo 'agent-7' > mount/.info/user` |
Examples below use `mount/` as the mount root. Shell users can substitute `/mnt/db/` or your actual mountpoint. Agent examples use tool-style paths (`Write`, `Glob`, `Bash`) from the bundled skill pack.
## Recipe map
| # | Name | Format | Primary tools |
|---|------|--------|---------------|
| 1 | Safe agent exploration | `markdown,history` | `.savepoint/`, `.undo/to-savepoint/` |
| 2 | Compare approaches | `markdown,history` | Savepoint + undo-to-savepoint |
| 3 | Multi-agent selective undo | `markdown,history` | `--user-id`, `.log/.by/user_id/`, per-user `.apply` |
| 4 | Task board (kanban) | `markdown,history` | `mkdir`, `mv`, `Glob`, `Grep` |
| 5 | Knowledge base | `markdown,history` | Topic dirs, `Grep`, `.history/` |
| 6 | Session context | `markdown` | Dated `.md` files, `status` frontmatter |
| 7 | Activity log | `markdown` | Append-only timestamped files |
Workflow patterns (1–3) cover safe editing; application patterns (4–7) cover durable workspaces agents build on mounts.
---
## Workflow patterns
### Recipe 1: Safe agent exploration
Create a named savepoint before uncertain work, explore, preview, then keep or revert.
```bash
echo '{"description":"Before investigating bug #42"}' > mount/notes/.savepoint/before-investigation.json
```
Savepoint files need a format suffix (`.json` preferred). With `--user-id` set, `user_id` is injected automatically.
Edit, create, or delete files under the workspace as usual.
```bash
cat mount/notes/.undo/to-savepoint/before-investigation/.info/summary
cd mount/notes && diff -ru .undo/to-savepoint/before-investigation . -x '.*'
```
Keep: do nothing (changes are already committed).
Revert (destructive — confirm with the user first):
```bash
touch mount/notes/.undo/to-savepoint/before-investigation/.apply
```
**Auto-savepoints:** After an inactivity gap (default **30 minutes**), the next write creates `auto--` (or `auto-` without user id). List with `Glob "mount/notes/.savepoint/auto-*"`. Disable with `--auto-savepoint-interval 0`; override default with e.g. `--auto-savepoint-interval 15m`.
### Recipe 2: Compare approaches with savepoints
Branch implementations without losing either result.
```bash
# Baseline
echo '{"description":"Clean baseline"}' > mount/app/.savepoint/baseline.json
# Implement approach A, then bookmark
echo '{"description":"DFS implementation complete"}' > mount/app/.savepoint/after-dfs.json
# Reset to baseline for approach B
touch mount/app/.undo/to-savepoint/baseline/.apply
# Implement B (now current)
# Compare: read summary for A's bookmarked state
# mount/app/.undo/to-savepoint/after-dfs/.info/summary
# Keep B: no action. Keep A instead:
touch mount/app/.undo/to-savepoint/after-dfs/.apply
```
Undo-to-savepoint restores the tree **as of that savepoint**, including work done after it (e.g. approach B), so you can switch winners without manual file surgery.
### Recipe 3: Multi-agent selective undo
Separate mounts or identities tag operations in `.log/`; undo can filter by `user_id`.
```bash
# Distinct identities (same database, different mounts)
tigerfs mount --user-id agent-research postgres://... /mnt/research
tigerfs mount --user-id agent-implement postgres://... /mnt/implement
# Shared sprint bookmark
echo '{"description":"Sprint start"}' > mount/notes/.savepoint/sprint-start.json
# Inspect one agent's recent ops
# Read mount/notes/.log/.by/user_id/agent-research/.last/10/.export/json
# Undo only that agent's changes since the savepoint
touch mount/notes/.undo/to-savepoint/sprint-start/.by/user_id/agent-research/.apply
```
If two agents edit the **same file**, per-user undo reverts the file to its state before that agent's **first** edit on it — which can also remove the other agent's interleaved edits on that file. Use separate files or savepoints when agents share paths.
---
## Application patterns
### Recipe 4: Task board (kanban)
Use for todo lists, kanban, project trackers, and shared queues. **Directories are states; files are items; `mv` is the transition; `author` tracks ownership.**
Do **not** model column state with a `status` frontmatter field. State lives in the directory path (`todo/`, `doing/`, `done/`). The bundled `skills/tigerfs/SKILL.md` enforces this for agents building boards.
#### Setup
```bash
echo 'markdown,history' > mount/.build/tasks
mkdir mount/tasks/todo mount/tasks/doing mount/tasks/done
```
#### Lifecycle
```text
mount/tasks/
├── todo/ ← pending
├── doing/ ← in progress (claim via mv)
└── done/ ← completed
```
```mermaid
stateDiagram-v2
[*] --> todo: Write new .md
todo --> doing: mv (claim)
doing --> done: mv (complete)
done --> [*]
```
Renaming a markdown file updates the `filename` column in the backing `tigerfs` table (same as `mv` on disk).
#### Add a task
```markdown
---
title: Fix Auth Bug
priority: high
---
The login endpoint returns 500 when session cookie is expired.
```
Path: `mount/tasks/todo/fix-auth-bug.md`
#### Claim and complete
```bash
mv mount/tasks/todo/fix-auth-bug.md mount/tasks/doing/fix-auth-bug.md
# optional: set author: your-name in frontmatter
mv mount/tasks/doing/fix-auth-bug.md mount/tasks/done/fix-auth-bug.md
```
#### Inspect the board
```bash
ls mount/tasks/todo/*.md
ls mount/tasks/doing/*.md
ls mount/tasks/done/*.md
grep -r "author:" mount/tasks/doing/*.md
```
#### Custom columns
Any directory name works: `backlog/`, `sprint/`, `review/`, `shipped/`. Add columns with `mkdir`; move cards with `mv`.
#### History
```bash
ls mount/tasks/.history/doing/fix-auth-bug.md/
```
Shows moves and edits when `history` was enabled at workspace creation.
### Recipe 5: Knowledge base with history
Topic directories replace tags-as-state; moving a file recategorizes it.
#### Setup
```bash
echo 'markdown,history' > mount/.build/kb
mkdir mount/kb/architecture mount/kb/debugging mount/kb/conventions
```
#### Store a decision
```markdown
---
title: Chose JWT Over Server Sessions
author: alice
confidence: high
---
## Decision
Use JWT tokens instead of server-side sessions.
## Reasoning
- Stateless -- no session store
- Works across multiple server instances
```
Path: `mount/kb/architecture/chose-jwt.md`
#### Recategorize
```bash
mv mount/kb/debugging/null-bytes.md mount/kb/conventions/null-bytes.md
```
#### Search
```bash
grep -r "authentication" mount/kb/
grep -r "confidence: high" mount/kb/*.md
```
#### Suggested frontmatter (extra headers / columns)
| Key | Example values | Purpose |
|-----|------------------|---------|
| `confidence` | `high`, `medium`, `low` | Certainty |
| `source` | free text | Provenance |
| `supersedes` | filename | Replaces an older note |
Keys without dedicated columns are stored in the `headers` JSONB column (full-replace on each write). Known columns like `title` and `author` keep old values when omitted.
#### Track evolution
```bash
ls mount/kb/.history/architecture/chose-jwt.md/
```
### Recipe 6: Session context (resuming work)
Lightweight continuity across agent sessions — uses **`status` frontmatter** here (unlike kanban).
#### Setup
```bash
echo 'markdown' > mount/.build/sessions
```
#### End of session
```markdown
---
title: Auth Refactor
status: in-progress
---
## Completed
- Migrated to JWT
- Updated /src/auth/middleware.ts
## Next Steps
- Implement refresh token rotation
```
Path: `mount/sessions/2026-02-24-auth-refactor.md` (date + topic naming).
#### Start of next session
```bash
ls mount/sessions/*.md
grep -r "status: in-progress" mount/sessions/*.md
cat mount/sessions/2026-02-24-auth-refactor.md
```
History is optional for session notes; add `,history` to the build format if you need versioned session files.
### Recipe 7: Activity log
Append-only, one file per event; safe for concurrent writers.
#### Setup
```bash
echo 'markdown' > mount/.build/activity
```
#### Log an event
Filename pattern: `YYYY-MM-DDTHHMMSS.mmmZ-short-description.md`
```markdown
---
author: agent-a
type: fix
---
Fixed the auth bug in login endpoint. Changed session cookie handling to check expiry before validating.
```
Path example: `mount/activity/2026-03-21T150000.000Z-fixed-auth-bug.md`
#### Review
```bash
ls mount/activity/*.md
ls mount/activity/2026-03-21*
grep -r "author: agent-a" mount/activity/
grep -r "type: fix" mount/activity/
```
---
## Format strings reference
| Build command | Workspace behavior |
|---------------|-------------------|
| `echo 'markdown' > mount/.build/name` | YAML frontmatter + body |
| `echo 'markdown,history' > mount/.build/name` | Above + `.history/`, `.log/`, `.savepoint/`, `.undo/` |
| `echo 'plaintext' > mount/.build/name` | Body only, no frontmatter |
| `echo 'history' > mount/.build/name` | Add history to existing workspace |
Backing tables live under `mount/.tables//` for data-first export, indexes, and schema inspection.
---
## Agent vs shell command styles
The skill pack documents two equivalent styles:
| Action | Agent (skill pack) | Shell |
|--------|-------------------|-------|
| Create workspace | `Bash "echo 'markdown,history' > mount/.build/tasks"` | `echo "markdown,history" > /mnt/db/.build/tasks` |
| List pending | `Glob "mount/tasks/todo/*.md"` | `ls /mnt/db/tasks/todo/` |
| Transition card | `Bash "mv mount/tasks/todo/x.md mount/tasks/doing/x.md"` | `mv /mnt/db/tasks/todo/x.md /mnt/db/tasks/doing/x.md` |
| Revert workspace | `Bash "touch mount/notes/.undo/to-savepoint/name/.apply"` | `touch /mnt/db/notes/.undo/to-savepoint/name/.apply` |
Natural-language requests ("set up a kanban with todo/doing/done") map to Recipe 4 via the bundled `skills/tigerfs/SKILL.md` routing table — no need to memorize paths if the agent has TigerFS skills installed at mount time.
---
## Verification signals
| Check | Expected |
|-------|----------|
| Workspace exists | `ls mount/tasks/` shows `todo/`, `doing/`, `done/` |
| Card moved | File only under new column directory; `mv` preserves frontmatter and body |
| Savepoint listed | `ls mount/notes/.savepoint/` includes your `.json` name |
| Undo preview | `.undo/to-savepoint//.info/summary` describes affected files |
| Per-user log | `.log/.by/user_id//.last/N/.export/json` returns JSON rows |
| Auto-savepoint | After gap + write, `auto-*` entry appears under `.savepoint/` |
On failure, TigerFS logs structured hints to stderr (errno alone is often insufficient). Use `tigerfs mount --debug` for full operation traces.
---
## Related pages
Create workspaces, frontmatter semantics, directories as state, and `.tables/` access.
`.history/`, `.log/`, savepoint JSON, preview/apply undo, and per-user filters.
Bundled `skills/tigerfs` pack, SKILL.md routing to recipes, mount-time install.
`.build/` formats, backing tables in `tigerfs` schema, and column mapping.
First mount, `ls`/`cat`, and demo workflows.
---
## 23. Migrate workspaces
> tigerfs migrate --describe/--dry-run, history-format migrations, pre-boundary undo EPERM, and release verification against test/integration migrate tests.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/23-migrate-workspaces.md
- Generated: 2026-06-03T08:02:17.214Z
### Source Files
- `internal/tigerfs/cmd/migrate.go`
- `test/integration/migrate_test.go`
- `docs/release-process.md`
- `docs/adr/019-undo-boundary-via-metadata-table.md`
- `internal/tigerfs/fs/undo_boundary_test.go`
---
title: "Migrate workspaces"
description: "tigerfs migrate --describe/--dry-run, history-format migrations, pre-boundary undo EPERM, and release verification against test/integration migrate tests."
---
The `tigerfs migrate` command (`internal/tigerfs/cmd/migrate.go`) connects to a PostgreSQL database, runs an ordered list of self-detecting migrations per user schema, and supports inspection (`--describe`), SQL preview (`--dry-run`), or transactional execution. Migrations upgrade synth workspaces from older TigerFS layouts (user-schema `_app` tables, path-encoded filenames, missing triggers) to the current model: backing tables in `tigerfs`, `parent_id` directories, and optional undo-boundary metadata after the `relational-directories` step.
## When to run migrate
Run migrate when a database was created with an older TigerFS release and still has pending structural changes. Fresh workspaces built via `/.build/` on a current release typically need no migration.
| Signal | Meaning |
|--------|---------|
| `tigerfs migrate … --describe` lists migrations | Work is pending |
| Output is `No pending migrations.` | Schema matches current TigerFS |
| Mount works but nested paths or undo behave oddly on an old DB | Likely skipped `relational-directories` |
| Undo of old log entries fails with permission errors after upgrade | Expected post `history-format-migration` boundary |
Migrate runs DDL in transactions per migration. Take a database backup before applying on production data. Use `--dry-run` first on shared or production databases.
## Command reference
```bash
tigerfs migrate [CONNECTION] [--describe] [--dry-run] [--schema SCHEMA] [--insecure-no-ssl]
```
Optional connection. Prefix selects backend: `tiger:ID`, `ghost:ID`, or `postgres://…`. If omitted, resolution uses config, env, and `PG*` variables (same as other TigerFS commands).
List pending migration names, summaries, and affected workspace/table identifiers. Does not execute SQL.
Print planned SQL (semicolon-terminated statements) without executing. Database state is unchanged.
User schema to scan (default: database `current_schema()` / `search_path`).
Allow non-TLS remote connections (same semantics as other TigerFS commands).
Default mode (no flags) executes each pending migration inside a single transaction, prints `Running migration: `, then `Migrated N views` per migration. Command timeout is 60 seconds.
```bash
tigerfs migrate postgres://user@host/dbname --describe
```
Example output shape:
```text
move-backing-tables: Move backing tables from user schema to tigerfs schema
- _myapp
relational-directories: Upgrade directory structure for improved performance and undo support
- myapp
```
```bash
tigerfs migrate postgres://user@host/dbname --dry-run
```
Expect `CREATE SCHEMA`, `ALTER TABLE … SET SCHEMA`, `ALTER TABLE … RENAME`, view recreation, and—for `relational-directories`—`parent_id`, history/log column renames, and metadata `INSERT` for the undo boundary.
```bash
tigerfs migrate postgres://user@host/dbname
```
Re-run `--describe` until you see `No pending migrations.`
## Migration pipeline
Migrations run in a fixed order; each has `Detect` (catalog queries) and `Plan` (SQL generation). Later migrations assume earlier ones completed where relevant.
```text
┌─────────────────────┐ ┌──────────────────────────┐ ┌────────────────────────────┐
│ move-backing-tables │ ──► │ relational-directories │ ──► │ parent-dir-mtime-trigger │
│ user._app → tigerfs │ │ parent_id, history/log │ │ parent directory mtime │
│ .app view │ │ metadata boundary row │ │ AFTER trigger on children │
└─────────────────────┘ └──────────────────────────┘ └────────────────────────────┘
```
| Name | Summary | Detect criteria (abbrev.) |
|------|---------|---------------------------|
| `move-backing-tables` | Move `_name` backing tables to `tigerfs.name`, recreate user-schema view | `_name` table + `tigerfs:` view comment with synth format; not already in `tigerfs` |
| `relational-directories` | Path-encoded filenames → `parent_id` model; history/log column renames; metadata table + boundary | `tigerfs` backing table exists, no `parent_id` column |
| `parent-dir-mtime-trigger` | POSIX-style directory `modified_at` on child changes | Has `parent_id`, missing `trg__parent_mtime` |
All three are idempotent: detection skips already-migrated objects; re-running with nothing pending prints `No pending migrations.`
### move-backing-tables
Targets legacy layout: backing table `_` in the user schema and view `` with a `tigerfs:` comment. For each match, migration:
1. Ensures `tigerfs` schema exists
2. Moves and renames `_` → `tigerfs.`
3. Drops and recreates the user-schema view pointing at `tigerfs`
4. If `_history` exists in the user schema, moves and renames it to `tigerfs._history`
Skips views whose comment resolves to native format without history (`FormatNative && !History`).
### relational-directories
Upgrades apps already in `tigerfs` (post move-backing-tables) from ADR-011 path-encoded filenames to ADR-017 `parent_id` directories. Per app, in one transaction:
- Adds `parent_id`, populates it from old path hierarchy, strips filenames to leaf names
- Switches `id` default to `uuidv7()`, adds FK and `UNIQUE NULLS NOT DISTINCT (parent_id, filename, filetype)`
- Recreates the user-schema view (PostgreSQL `SELECT *` views do not pick up new columns automatically)
- If history exists: renames `id`→`file_id`, `_history_id`→`version_id`, `_operation`→`operation`; maps `UPDATE`/`DELETE` to `edit`/`delete`; rebuilds archive trigger
- If log exists: renames `history_id`→`version_id`; maps log types `insert`→`create`, `update`→`edit`; updates CHECK constraint
- Creates `tigerfs._metadata` if missing and inserts exactly one `history-format-migration` row (`WHERE NOT EXISTS` guard)
The boundary `INSERT` runs at the end of this migration so its `entry_id` (UUIDv7) sorts after every pre-migration `log_id`. That lexical ordering is what the undo engine uses to block pre-boundary targets.
### parent-dir-mtime-trigger
Adds `GenerateParentDirMtimeTriggerSQL` for apps on the parent-pointer model so directory `modified_at` updates when children are inserted, deleted, or moved—important for NFS directory listing freshness.
## History-format migration and undo boundary
ADR-019 introduces `tigerfs._metadata` as a sibling of `_history`, `_log`, and `_savepoint`. The `relational-directories` migration inserts one row:
| Field | Value |
|-------|--------|
| `subject` | `history-format-migration` (`synth.SubjectHistoryFormatMigration`) |
| `description` | User-facing hint when undo is refused |
| `payload` | `{"from":"0.6","to":"0.7","reason":"parent-pointer model"}` |
**Why undo is blocked:** migration sets history `parent_id` from the *current* source row, not the historical directory at edit time. Pre-migration log entries remain structurally valid but undo could restore content while leaving the file in the wrong directory.
```mermaid
sequenceDiagram
participant User
participant Mount as fs.Operations / mount
participant Undo as undo.go checkBoundary
participant Meta as tigerfs.app_metadata
User->>Mount: .undo/.../.apply (target log_id)
Mount->>Undo: checkBoundary(schema, app, targetLogID)
Undo->>Meta: cached metadata rows (mount lifetime)
alt targetLogID < blocker.entry_id
Undo-->>Mount: FSError ErrPermission + Hint=description
Mount-->>User: EPERM (FUSE/NFS) + stderr hint
else targetLogID >= blocker.entry_id
Undo-->>Mount: nil (proceed to undo SQL)
end
```
### Pre-boundary undo: EPERM and hints
`checkBoundary` in `internal/tigerfs/fs/undo.go` runs before `ExecuteUndoSingle`, `ExecuteUndoToLogID`, and `ExecuteUndoToSavepoint` database work. Rules:
| Condition | Result |
|-----------|--------|
| No metadata rows (fresh 0.7+ install) | Allow undo (fast path) |
| `targetLogID < entry_id` for a blocking subject | Refuse: `ErrPermission`, `Hint` = row `description` |
| `targetLogID == entry_id` | Allow (boundary row itself is not a blocked target) |
| `targetLogID > entry_id` | Allow |
Blocking subjects are hard-coded in `blockingSubjects` (currently only `history-format-migration`). FUSE maps `ErrPermission` to `syscall.EPERM`; NFS maps to `os.ErrPermission`.
```bash
# After upgrading a v0.6 workspace — undo of an old log id
touch /mnt/db/myapp/.undo/id//.apply
```
```text
# stderr (structured logging) includes the metadata description as hint
# shell
apply: Operation not permitted # EPERM
```
Pre-migration history remains readable via `.log/` and `.history/`; only `.undo/` apply paths refuse. Post-migration edits, savepoints, and undo-to-log-id targets after the marker behave normally (covered in `verifyMigrationUndoBoundary` in integration tests).
Fresh installs: `/.build/` creates an empty `_metadata` table. With zero rows, `checkBoundary` returns immediately and undo is unrestricted (`TestSynth_FreshInstall_MetadataFastPath`).
## Release verification
When a release adds or changes migrations, extend the release checklist (`docs/release-process.md` §1b):
```bash
docker run --rm -d --name migrate-test -e POSTGRES_PASSWORD=test \
-p 15433:5432 timescale/timescaledb-ha:pg18
sleep 5
```
Use a database snapshot or SQL fixture representing the previous release layout before running migrate.
```bash
CONN='postgres://postgres:test@127.0.0.1:15433/postgres?sslmode=disable'
tigerfs migrate "$CONN" --describe
tigerfs migrate "$CONN" --dry-run
tigerfs migrate "$CONN"
```
```bash
TEST_DATABASE_URL="$CONN" go test -count=1 -timeout 600s ./test/integration/... -run '^TestSynth_Migrate'
```
Tests in `test/integration/migrate_test.go`:
| Test | Covers |
|------|--------|
| `TestSynth_MigrateDetectAndExecute` | `move-backing-tables`: describe, dry-run, execute, idempotency |
| `TestSynth_MigrateWithHistory` | Backing + `_history` table move |
| `TestSynth_MigrateAddParentPointer` | `relational-directories`, FS ops on migrated tree, undo boundary block/allow |
| `TestSynth_MigrateAddParentDirMtimeTrigger` | Trigger install and mtime behavior |
| `TestSynth_MigrateParentDirMtimeTrigger_NotNeeded` | Fresh build skips trigger migration |
| `TestSynth_FreshInstall_MetadataFastPath` | Empty metadata, undo allowed |
Unit tests for boundary logic: `go test ./internal/tigerfs/fs/... -run TestCheckBoundary` (`undo_boundary_test.go`).
For v0.7-style releases, manually confirm pre-boundary `.undo/…/.apply` returns EPERM with the metadata `description` as the logged hint.
`docs/release-process.md` suggests `-run TestMigrate`; the repository’s migrate integration tests use the `TestSynth_Migrate` prefix. Use `-run '^TestSynth_Migrate'` (or a narrower name) to match actual test names.
## Troubleshooting
| Symptom | Likely cause | Action |
|---------|----------------|--------|
| `No pending migrations.` but mount errors on old DB | Wrong `--schema` | Pass `--schema` for the schema that owns `tigerfs:` views |
| `migration …: failed to execute SQL` | DDL conflict, permissions, or extension missing | Read failed SQL in error; fix DB state; restore from backup if needed |
| Undo EPERM with long hint after upgrade | Pre-boundary log target | Use `.log//before` or `.history/`; do not `.undo/` pre-migration ids |
| Duplicate boundary rows | Should not happen | `INSERT … WHERE NOT EXISTS` keeps one marker; if count ≠ 1, inspect `tigerfs._metadata` manually |
| `relational-directories` not listed | Tables still in user schema | Run `move-backing-tables` first (pipeline order) |
| History-enabled app missing TimescaleDB | Hypertable expectations | Install TimescaleDB extension before migrate on history workspaces |
## Related pages
`.log`, `.history`, `.undo`, savepoints, and how the migration boundary interacts with undo.
Full `migrate` entry alongside mount, build, and connection commands.
Workspace layout before and after `parent_id` migration.
`go test`, integration testcontainers, and pre-commit checks.
Mapping `fs.ErrPermission` to EPERM and reading stderr hints.
---
## 24. Develop and test
> Build commands, go test ./... and integration testcontainers, pre-commit checks, stress runner, and release-process checklist.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/24-develop-and-test.md
- Generated: 2026-06-03T08:02:22.164Z
### Source Files
- `CLAUDE.md`
- `docs/release-process.md`
- `test/integration/main_test.go`
- `test/stress/README.md`
- `scripts/pre-commit`
- `.github/workflows/test.yml`
---
title: "Develop and test"
description: "Build commands, go test ./... and integration testcontainers, pre-commit checks, stress runner, and release-process checklist."
---
TigerFS is a Go module (`github.com/timescale/tigerfs`, Go 1.25.1) built from `cmd/tigerfs` into `bin/tigerfs`. Local verification spans unit tests under `internal/tigerfs/`, mount-based integration tests in `test/integration/` (PostgreSQL via local env or testcontainers), optional Docker FUSE runs via `scripts/test-docker.sh`, the `tigerfs-stress` binary in `test/stress/`, and a release checklist in `docs/release-process.md` that pairs Postgres 18 + TimescaleDB with GoReleaser tagging.
## Prerequisites
| Requirement | Used by |
|-------------|---------|
| Go 1.25+ (see `go.mod`) | Build, all `go test` |
| PostgreSQL / Docker | DB unit tests (`internal/tigerfs/db/`), integration, release |
| Linux: FUSE (`/dev/fuse`, `libfuse-dev`) | Native integration mounts, `scripts/test-docker.sh`, CI |
| macOS: NFS (`/sbin/mount_nfs`) | Default integration mount on Darwin |
| Docker daemon | testcontainers fallback, stress runner, docker-FUSE stress |
Many `internal/tigerfs/db/` tests call `t.Skip` when `PGHOST` / `TEST_DATABASE_URL` are unset. CI and release flows always set those variables so DB tests actually run.
## Build
Primary binary:
```bash
go build -o bin/tigerfs ./cmd/tigerfs
```
Build all packages (matches CI):
```bash
go build -v ./...
```
Stress runner (separate main under `test/stress`):
```bash
go build -o bin/tigerfs-stress ./test/stress
```
Release snapshot (no tag push):
```bash
goreleaser release --snapshot --clean
./dist/tigerfs_darwin_arm64*/tigerfs version
```
GoReleaser builds `tigerfs` for `linux` and `darwin` × `amd64`/`arm64` with `CGO_ENABLED=0`, embedding version metadata via ldflags.
## Unit tests
### Full tree
```bash
go test ./... # everything, including integration
go test -race ./... # race detector
go test -run TestName ./path # single test
```
### Package-scoped (no integration)
Exclude mount tests when iterating quickly:
```bash
PACKAGES=$(go list ./... | grep -v /test/integration)
go test -race $PACKAGES
```
### DB-connected unit tests
Point at a running Postgres (CI uses port 5432; release doc uses 15432):
```bash
PGHOST=127.0.0.1 PGPORT=5432 PGUSER=postgres PGPASSWORD=test PGDATABASE=postgres \
TEST_DATABASE_URL='postgres://postgres:test@127.0.0.1:5432/postgres?sslmode=disable' \
go test -count=1 -timeout 300s ./internal/tigerfs/...
```
`-count=1` disables the Go test cache so skipped tests do not silently stay skipped from a prior run.
### Synth tests
Filter all synthesized-app tests across packages:
```bash
go test ./internal/tigerfs/fs/synth/...
go test -run "^TestSynth_" ./internal/tigerfs/fs/... ./test/integration/...
```
FUSE-layer tests that need a live mount are skipped in-package with a pointer to `test/integration/`; filesystem behavior belongs in integration tests.
## Integration tests
Package: `test/integration/`. Tests mount TigerFS against real PostgreSQL and exercise `os.ReadFile`, `os.Rename`, pipelines, synth workspaces, DDL, history/undo, and more.
### Database provisioning
`TestMain` in `main_test.go`:
1. Probes local PostgreSQL (`TEST_DATABASE_URL` or `PG*` vars, with Unix-socket detection when no host/password).
2. If local PG is unavailable and Docker is reachable, starts one shared **TimescaleDB HA pg18** container via testcontainers-go.
3. Each test isolates data with unique schemas (`setupLocalTestDB` / `setupLocalTestDBEmpty`).
```bash
go test -count=1 -timeout 600s ./test/integration/...
go test -count=1 -timeout 600s ./test/integration/... -run TestMount_ListTables
```
If neither local PG nor Docker is available, tests skip with `Neither local PostgreSQL nor Docker available`.
### Mount method matrix
| Host OS | Default mount | Override |
|---------|---------------|----------|
| Linux | FUSE (`/dev/fuse`) | — |
| macOS | NFS (in-process `go-nfs`) | `TEST_MOUNT_METHOD=docker` runs tests in a Linux FUSE container |
Platform-specific test prefixes (use only when behavior is mount-specific):
| Prefix | Meaning |
|--------|---------|
| `TestMount_`, `TestWrite_`, `TestDDL_`, … | Generic; runs on FUSE or NFS |
| `TestNFS_` | NFS-only; skipped on Linux/FUSE |
| `TestFUSE_` | FUSE-only; skipped on macOS/NFS |
### Docker FUSE integration (Linux container)
For Linux-native FUSE from any host (including macOS):
```bash
./scripts/test-docker.sh # all integration tests
./scripts/test-docker.sh -run TestPipeline # filter
./scripts/test-docker.sh -v -timeout 10m # extra go test flags
./scripts/test-docker.sh -out results.log # custom log path
```
Uses `test/docker/docker-compose.test.yml`; tears down compose on exit.
## Pre-commit checks
Install the git hook:
```bash
./scripts/install-hooks.sh
```
Hook script `scripts/pre-commit` runs on every commit (skip with `git commit --no-verify`):
`gofmt -l .` must be empty (otherwise run `go fmt ./...`).
`go vet ./...`
`go test -race` on all packages **except** `test/integration`.
Full maintainer gate before commit (from project guidance — includes integration and mod tidy):
```bash
go fmt ./... && go vet ./... && go test ./... && go mod tidy
```
The pre-commit hook does **not** run integration tests or `go mod tidy`. Release and CI cover those surfaces.
## CI workflows
### `test.yml` (default on PR and `main` push)
| Step | Command / behavior |
|------|------------------|
| Service | `timescale/timescaledb:latest-pg18` on port 5432 |
| FUSE | `apt-get install fuse libfuse-dev` |
| Tidy check | `go mod tidy` + `git diff --exit-code go.mod go.sum` |
| Build | `go build -v ./...` |
| Vet | `go vet ./...` |
| Unit tests | `-race -timeout 300s -coverprofile=coverage.txt`, packages excluding `test/integration`, with `PGHOST` / `TEST_DATABASE_URL` set |
| Integration | Only when `workflow_dispatch` input `run_integration: true` |
| Coverage | Codecov upload on PR / `main` |
Trigger integration tests manually:
```bash
gh workflow run test.yml --ref "$(git branch --show-current)" -f run_integration=true
```
### `stress.yml`
Path-filtered on PRs touching `fs/`, `db/`, `fuse/`, migrate, or stress scripts. Runs `./scripts/stress-docker.sh` on Ubuntu with FUSE inside Docker:
| Trigger | Iterations |
|---------|------------|
| PR | 200 |
| Weekly schedule (Mon 06:17 UTC) | 1000 |
| `workflow_dispatch` | Configurable input |
Flags: `--validate-every 1 --large-files --many-files`. Failure dumps upload from `test/stress/docker/host-out/`.
### `release.yml`
Pushing a `v*.*.*` tag runs GoReleaser `release --clean` and publishes binaries; non-prerelease tags also sync `install.sh` and `latest.txt` to S3.
## Stress runner (`tigerfs-stress`)
Standalone binary — **no TigerFS internal imports**. It drives a mounted workspace through `os.*` calls, maintains an in-memory MD5 model, and validates after every op (and around undo).
```text
tigerfs-stress ── os.WriteFile ──> kernel mount ──> TigerFS ──> PostgreSQL
│ │
└── ValidateWorkspace() ◄── os.ReadFile + hash compare
```
| Mode | Where tigerfs runs | Mount |
|------|-------------------|-------|
| `bin/tigerfs-stress start` on macOS | Host | NFS |
| Same on Linux | Host | FUSE |
| `./scripts/stress-docker.sh` | Linux container | FUSE |
Common commands:
```bash
bin/tigerfs-stress start
bin/tigerfs-stress start --seed 42 --iterations 50
bin/tigerfs-stress start --large-files --many-files --iterations 100 --validate-every 5
bin/tigerfs-stress stop
./scripts/stress-docker.sh --seed 42 --iterations 200
```
Stress package unit tests (no Docker/TigerFS):
```bash
go test ./test/stress/...
```
Long macOS NFS runs may print `[warn iter ...] readLatestLogID regressed` lines. The runner retries and continues; these reflect NFS snapshot timing, not TigerFS data loss. See the troubleshooting page for interpretation.
Exit codes: `0` pass, `1` verification failure, `2` infrastructure failure. Validation failures write diagnostic dumps under `/tmp/tigerfs-stress-*` (or `test/stress/docker/host-out/` in docker mode) and leave infrastructure up for inspection.
## Release checklist
Follow `docs/release-process.md` end-to-end before tagging.
```bash
docker run --rm -d --name tigerfs-release-pg -e POSTGRES_PASSWORD=test \
-p 15432:5432 timescale/timescaledb-ha:pg18
sleep 5
```
```bash
go fmt ./... && go vet ./... && go mod tidy
PGHOST=127.0.0.1 PGPORT=15432 PGUSER=postgres PGPASSWORD=test PGDATABASE=postgres \
TEST_DATABASE_URL='postgres://postgres:test@127.0.0.1:15432/postgres?sslmode=disable' \
go test -count=1 -timeout 300s ./internal/tigerfs/...
go test -count=1 -timeout 600s ./test/integration/...
./scripts/test-docker.sh -v -timeout 300s
```
Seed a pre-release workspace, then `tigerfs migrate --describe`, `--dry-run`, apply, and re-run `go test ... -run TestMigrate` against the migrated DB.
Add a user-facing section to `CHANGELOG.md` (replace `YYYY-MM-DD` with the release date). Mark tasks in `docs/implementation/implementation-tasks-checklist.md`.
`goreleaser release --snapshot --clean` and smoke-test `./dist/.../tigerfs version`.
```bash
git add CHANGELOG.md README.md docs/
git commit -m "docs: prepare vX.Y.Z release"
git tag vX.Y.Z
git push origin main
git push origin vX.Y.Z
```
Edit the GitHub release body to match `CHANGELOG.md` (GoReleaser auto-changelog is a starting point only).
`docker stop tigerfs-release-pg`
## Quick reference
| Goal | Command |
|------|---------|
| Build CLI | `go build -o bin/tigerfs ./cmd/tigerfs` |
| Fast unit loop | `go test -race $(go list ./... \| grep -v /test/integration)` |
| Integration (local) | `go test -count=1 -timeout 600s ./test/integration/...` |
| Integration (FUSE in Docker) | `./scripts/test-docker.sh` |
| Install pre-commit hook | `./scripts/install-hooks.sh` |
| Stress (native) | `go build -o bin/tigerfs-stress ./test/stress && bin/tigerfs-stress start` |
| Stress (FUSE in Docker) | `./scripts/stress-docker.sh` |
| Release snapshot | `goreleaser release --snapshot --clean` |
```mermaid
flowchart TB
subgraph local["Local developer"]
fmt["go fmt / go vet"]
unit["go test internal/..."]
integ["go test test/integration/..."]
docker["scripts/test-docker.sh"]
stress["bin/tigerfs-stress"]
end
subgraph pg["PostgreSQL"]
localPG["Local PG / PG* env"]
tc["testcontainers shared container"]
end
subgraph ci["GitHub Actions"]
testyml["test.yml"]
stressyml["stress.yml"]
rel["release.yml on v* tag"]
end
fmt --> unit
unit --> localPG
integ --> localPG
integ --> tc
docker --> integ
stress --> pg
testyml --> unit
stressyml --> stress
rel --> rel
```
## Related pages
Platform prerequisites (FUSE, NFS, Docker) before running tests locally.
Stress monotonicity warnings, `--debug` logging, and errno recovery.
Release-time `tigerfs migrate` verification and `TestMigrate` integration coverage.
`demo.sh` workflows for manual exploration separate from automated tests.
---
## 25. Troubleshooting
> Stale mounts, force unmount, NFS monotonicity warnings in stress runs, logging --debug, .refresh metadata clear, and common errno recovery.
- Page Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/pages/25-troubleshooting.md
- Generated: 2026-06-03T08:03:54.747Z
### Source Files
- `internal/tigerfs/cmd/unmount.go`
- `internal/tigerfs/cmd/stop.go`
- `test/stress/README.md`
- `internal/tigerfs/logging/logging.go`
- `docs/spec.md`
- `CHANGELOG.md`
---
title: "Troubleshooting"
description: "Stale mounts, force unmount, NFS monotonicity warnings in stress runs, logging --debug, .refresh metadata clear, and common errno recovery."
---
TigerFS surfaces failures through POSIX errno on the mount, structured JSON logs on stderr, and a JSON mount registry under `~/.config/tigerfs/mounts.json`. Most operational issues—stale registry rows, busy unmounts, metadata that lags external DDL, stress-run NFS warnings, and opaque `EINVAL`/`EACCES` from shell tools—are diagnosed from those three channels plus the `tigerfs` lifecycle commands (`status`, `list`, `unmount`, `stop`).
## Symptom map
| Symptom | First checks | Typical fix |
|---------|--------------|-------------|
| `tigerfs status` shows a mount you killed | Registry PID stale | `tigerfs status` or `list` (auto-`Cleanup`) |
| `unmount: device is busy` | Open files under mountpoint | Close processes, then `tigerfs unmount --force` |
| `ls` missing a new table | Metadata cache TTL | Wait for `metadata_refresh_interval`, or remount |
| Tool reports `EINVAL` with no detail | TigerFS stderr | Re-run with `--log-level debug` |
| `[warn iter N] readLatestLogID regressed` | Stress run on macOS NFS | Expected; see monotonicity section |
```text
shell tool (mv, cat, ls)
│
▼
FUSE / NFS adapter ──► errno (ENOENT, EINVAL, …)
│
▼
fs.Operations ──► FSError + zap hint on stderr
│
▼
PostgreSQL
```
## Stale mount registry entries
The mount registry records mountpoint, PID, database label, and start time. Entries survive crashes when TigerFS exits without `unmount` or `stop`.
`Register` replaces duplicate mountpoints when a prior process died without cleanup. `Cleanup()` removes entries whose PID is no longer running. `ListActive()` returns only live processes.
| Command | Registry behavior |
|---------|-------------------|
| `tigerfs status` | Calls `Cleanup()`, then lists active mounts |
| `tigerfs list` | Same cleanup; prints mountpoints only (scripting) |
| `tigerfs status /path/to/mount` | Shows `active` or `stale` for that entry’s PID |
| `tigerfs unmount MOUNT` | Unmounts, then `Unregister` (best-effort) |
```bash
tigerfs status
# or
tigerfs status /mnt/db
```
`active` means the PID responds to a signal probe. `stale` means the registry row outlived the process.
```bash
tigerfs list # cleanup + active mountpoints
tigerfs status # human-readable table
```
Both commands invoke `registry.Cleanup()` before listing.
If the directory still exists but nothing is mounted:
```bash
# macOS
diskutil unmount force /mnt/db 2>/dev/null || true
# Linux FUSE
fusermount -uz /mnt/db 2>/dev/null || true
```
`unmount` treats “not currently mounted” / “not mounted” as success.
A stale registry row does not mean the filesystem is still mounted. Check the mount with `mount | grep tigerfs` (Linux) or `mount` (macOS) before force-unmounting.
## Force unmount and stuck mounts
`unmount` tries graceful shutdown first: look up the registry entry, send `SIGTERM`, poll every 100ms until the context timeout (default 30s), then fall back to OS unmount.
| Platform | Normal | Force (`-f` / `--force`) |
|----------|--------|---------------------------|
| macOS | `umount MOUNT` | `diskutil unmount force MOUNT` |
| Linux | `fusermount -u MOUNT` | `fusermount -uz MOUNT` (lazy) |
| Other | `umount MOUNT` | `umount -f MOUNT` |
Seconds to wait for the TigerFS process to exit after `SIGTERM` before system unmount.
```bash
# Graceful (registry SIGTERM, then system unmount)
tigerfs unmount /mnt/db
# Busy mountpoint
tigerfs unmount --force /mnt/db
# Known PID instead of path
tigerfs stop 12345
```
`stop PID` sends `SIGTERM`, waits for exit, and unregisters the mountpoint if found in the registry. Use when you have the PID from `tigerfs status` but not the path.
Force unmount can strand open file handles in client processes. Close editors, shells with `cd` into the mount, and long-running jobs before `--force`.
### Stress-test teardown
`tigerfs-stress` uses the same platform rules: SIGTERM with a 5s grace window, then `Kill`, then `diskutil unmount force` (macOS) or `umount` (Linux). For a stuck stress run from another terminal:
```bash
bin/tigerfs-stress stop
# Docker FUSE mode (match info file env):
TIGERFS_STRESS_INFO_FILE=/out/tigerfs-stress.info \
docker compose -f test/stress/docker/docker-compose.yml \
exec -e TIGERFS_STRESS_INFO_FILE=/out/tigerfs-stress.info \
stress /usr/local/bin/tigerfs-stress stop
```
## Debug logging (`--log-level debug`)
FUSE and NFS adapters return errno only. TigerFS logs the underlying reason (and optional `hint` fields) to stderr before returning.
| Level | Config / flag | Output style |
|-------|---------------|--------------|
| `warn` (default) | `log_level: warn` | Minimal production JSON |
| `debug` | `--log-level debug` | Development: colors, timestamps, caller |
```bash
# Foreground mount with full diagnostics
tigerfs mount --foreground --log-level debug postgres://host/db /mnt/db
# Any subcommand inherits persistent flags
tigerfs --log-level debug status
# Config file or env (viper)
# ~/.config/tigerfs/config.yaml: log_level: debug
# export TIGERFS_LOG_LEVEL=debug
```
At `debug` or `info`, logging uses `zap.NewDevelopmentConfig` (colored levels, ISO8601 timestamps). At `warn` and `error`, output is compact production JSON with `message` as the primary key.
Stress runs pass debug through to the child TigerFS process:
```bash
bin/tigerfs-stress start --debug --iterations 10
# equivalent: --log-level debug on the tigerfs binary
```
TigerFS process stdout/stderr from stress land in `tigerfs.log` in the working directory.
Reproduce the failing operation with debug enabled on a foreground mount. Search stderr for `"hint"` — synth validation, DDL staging, and undo paths emit structured guidance there.
## Metadata cache refresh
Metadata (schemas, tables, views, PKs, permissions) is cached in-process with tiered TTLs. Row **content** is not cached; stale symptoms usually affect directory listings or stat sizes, not `cat` on a known path.
| Cache tier | Config key | Default TTL |
|------------|------------|-------------|
| Catalog (schemas, tables, views) | `metadata_refresh_interval` | 10s |
| Structural (row counts, permissions, PKs) | `structural_metadata_refresh_interval` | 5m |
| Stat / path (per-table listings) | Fixed in code | 2s |
| Undo preview | Fixed in code | 2s |
`MetadataCache.Invalidate()` clears all tiers and forces a DB refresh on next access. It runs after DDL commits, workspace build operations, and similar write paths.
### Manual clear via `.refresh`
`docs/spec.md` documents a mount-root control file:
```bash
echo 1 > /mnt/db/.refresh
```
Writing to `.refresh` is specified to clear cached metadata (table lists, columns, indexes, permissions). The active `fs.Operations` root listing currently exposes `.build`, `.create`, `.info`, `.schemas`, `.tables`, and `.views` — not `.refresh`. If `ls -a` at the mount root does not show `.refresh`, use the workarounds below instead of expecting that path to exist.
After external `CREATE TABLE` / `DROP` outside TigerFS, new names appear after `metadata_refresh_interval` (default 10s) unless a TigerFS write path already called `Invalidate()`.
```yaml
# ~/.config/tigerfs/config.yaml
metadata_refresh_interval: 5s
structural_metadata_refresh_interval: 2m
```
Or `TIGERFS_METADATA_REFRESH_INTERVAL` / `TIGERFS_STRUCTURAL_METADATA_REFRESH_INTERVAL` where supported by your config loader.
```bash
tigerfs unmount /mnt/db
tigerfs mount postgres://host/db /mnt/db
```
Direct file access (`cat /mount/table/row/col`) always queries PostgreSQL. If `cat` works but `ls` on the parent directory is wrong, suspect metadata or kernel directory cache (often 1–2s on FUSE), not row content staleness.
## NFS monotonicity warnings (stress runs)
On macOS, TigerFS serves the mount through an in-process NFS v3 server (`go-nfs`). The stress harness (`bin/tigerfs-stress`) reads `.log/.last/N/.export/json` over NFS. After heavy undo operations, the macOS NFS client can return a **stale** snapshot where the newest `log_id` is older than one already observed (UUIDv7 ordering makes that impossible for a true latest value).
The runner wraps reads in `readLatestLogIDMonotonic`:
- Up to **25** retries, **100ms** apart (~2.5s budget)
- On recovery: stderr warning `[warn iter N] readLatestLogID regressed after …; recovered after …`
- On exhaustion: keeps the prior known-good `lastLogID` and logs that recovery failed
These warnings are **expected** during stress runs. They indicate NFS/kernel attribute caching below TigerFS, not application data loss. Do not treat them as TigerFS bugs to fix; the run continues with the last known-good log ID.
End-of-run summary (stdout):
```text
=== Monotonicity Warnings ===
Total: N regressions across M ops (x.xx%)
Recovered: … / … (… kept prior lastLogID after retry exhaustion)
```
To compare NFS vs FUSE behavior, run the same seed natively on macOS (NFS) and via `./scripts/stress-docker.sh` (Linux FUSE in Docker).
| Observation | Interpretation |
|-------------|----------------|
| Warnings only on macOS native | NFS client / go-nfs path |
| Same seed fails validation | Use failure dump under `/tmp/tigerfs-stress-failure-*` or `test/stress/docker/host-out/` |
| Seed does not replay | NFS timing can diverge PRNG consumption; rely on dump artifacts |
## Common errno recovery
Shell tools show only errno. Map symptoms using stderr JSON and this table.
### `fs.ErrorCode` → POSIX (FUSE adapter)
| `FSError` code | errno | Typical cause |
|----------------|-------|----------------|
| `ErrNotExist` | `ENOENT` | Missing path, row, or staging entry |
| `ErrPermission` | `EACCES` | Grants, constraints, reserved dotfile |
| `ErrInvalidPath`, `ErrInvalidFormat`, `ErrInvalidArgument` | `EINVAL` | Bad path, malformed PATCH/JSON, synth numbering |
| `ErrInvalidOperation` | `EPERM` | Operation not allowed on path type |
| `ErrReadOnly` | `EROFS` | Read-only mount |
| `ErrNotEmpty` | `ENOTEMPTY` | `rmdir` on non-empty directory |
| `ErrAlreadyExists` | `EEXIST` | Create when path exists |
| `ErrIO`, `ErrInternal` | `EIO` | DB connection, timeout, unexpected SQL |
| `ErrNotImplemented` | `ENOSYS` | Unimplemented feature |
### PostgreSQL errors → errno (user-visible)
| Database condition | errno | Where detail lives |
|--------------------|-------|-------------------|
| Row not found | `ENOENT` | stderr at `debug` |
| Permission / NOT NULL / FK violation | `EACCES` | stderr `ERROR` with SQL detail |
| Connection refused / timeout | `EIO` | stderr + `tigerfs test-connection` |
```bash
# User sees only:
echo 'x' > /mnt/db/users/1/email
# email: Permission denied
# Check exit code
echo $? # 13 = EACCES on Linux
# TigerFS stderr (run mount with --log-level debug):
# {"message":"NOT NULL constraint violation",...,"hint":"..."}
```
### Recovery patterns
| errno | Check | Action |
|-------|-------|--------|
| `ENOENT` | Path spelling, PK segment, pipeline depth | `ls` parent; read stderr hint for synth paths |
| `EINVAL` | Format suffix (`.json`, `.tsv`), task numbers, undo order | stderr `hint`; undo **newest first** |
| `EACCES` | Table grants, NOT NULL, reserved `.name` | Fix SQL or pick non-reserved filename |
| `EPERM` | Read-only mount, history path, DDL policy | `tigerfs mount` without `--read-only` |
| `EIO` | Network, `sslmode`, pool exhaustion | `tigerfs test-connection URL`; TLS flags |
| `EBUSY` (unmount) | Processes using mount | `lsof +D /mnt/db`; `unmount --force` |
`ErrNotExist` is logged at **debug** level to avoid noise from tab completion and speculative `stat`. For missing-file investigations, always use `--log-level debug`.
### Cross-mount consistency
If mount A wrote data and mount B still shows wrong directory entries within ~2s, suspect stat/path cache TTL or metadata catalog TTL—not stale row reads. Writes should call `statCache.invalidate(schema, table)` and `pathCache.invalidate(schema, table)` with the **user** schema (not internal `tigerfs` schema keys). A schema-key mismatch leaves stale cache entries for up to 2 seconds.
## Quick reference
| Goal | Command |
|------|---------|
| List live mounts | `tigerfs status` |
| Scriptable mountpoints | `tigerfs list` |
| Graceful stop | `tigerfs unmount /mnt/db` |
| Force stop | `tigerfs unmount -f /mnt/db` |
| Stop by PID | `tigerfs stop PID` |
| Verbose server logs | `tigerfs --log-level debug mount --foreground …` |
| Stress debug | `bin/tigerfs-stress start --debug` |
| Stop stress infra | `bin/tigerfs-stress stop` |
## Related pages
Full errno mapping, constraint behavior, and reading stderr hints.
Cross-mount freshness rules, cache TTLs, and invalidation on writes.
`mount`, `unmount`, `stop`, `status`, `list`, and global flags.
`metadata_refresh_interval`, `log_level`, and env precedence.
Linux FUSE vs macOS NFS write/commit behavior.
Stress runner, integration tests, and pre-commit checks.
---