# Consistency and caching

> Cross-mount read freshness (no content cache), metadata/stat/path cache TTLs, invalidation on writes, and pipeline cache key isolation.

- Repository: timescale/tigerfs
- GitHub: https://github.com/timescale/tigerfs
- Human docs: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3
- Complete Markdown: https://grok-wiki.com/public/docs/timescale-tigerfs-60719456a5c3/llms-full.txt

## Source Files

- `CLAUDE.md`
- `internal/tigerfs/fs/metadata_cache.go`
- `internal/tigerfs/fs/path_cache.go`
- `docs/adr/014-query-reduction-caching.md`
- `internal/tigerfs/fs/operations.go`

---

---
title: "Consistency and caching"
description: "Cross-mount read freshness (no content cache), metadata/stat/path cache TTLs, invalidation on writes, and pipeline cache key isolation."
---

TigerFS treats PostgreSQL as the single source of truth across every mount of the same database: `ReadFile`, export reads, and column reads always query the database, while `Operations` in `internal/tigerfs/fs` caches only metadata (directory entries, stat sizes, catalog lists) with short TTLs and explicit invalidation on writes.

## Design principle

Multiple TigerFS processes can mount one database at the same time. The consistency contract is:

| Data class | Cached? | Cross-mount freshness |
|------------|---------|------------------------|
| Row/column/export **content** | Never | Always current (every read hits PostgreSQL) |
| Directory **listings** (`ReadDir`) | No result cache | Always current |
| **Stat** metadata (size, mode, mtime) | Yes, 2s TTL + write invalidation | Visible on other mounts within ~2s |
| **Path resolution** (parent → row ID) | Yes, 2s TTL + write invalidation | Same as stat cache |
| **Catalog** (schemas, tables, views) | Yes, configurable TTL | Default 10s; explicit invalidate on DDL/build |
| **Structural** (row counts, permissions, PK negatives) | Yes, slower TTL | Default 5m |
| **Column definitions** | Session cache until invalidate | Cleared with `MetadataCache.Invalidate()` |

<Warning>
Never cache row values or export payloads. A write on mount A must be visible to `cat` on mount B immediately. Only metadata may be stale for a bounded window.
</Warning>

## Cache tiers in `fs.Operations`

ADR-014 defines a three-tier query-reduction strategy. The shared `Operations` struct (`internal/tigerfs/fs/operations.go`) implements these plus auxiliary caches:

```mermaid
flowchart TB
  subgraph clients [Mount clients]
    FUSE[FUSE adapter]
    NFS[NFS adapter go-nfs]
  end

  subgraph ops [fs.Operations]
    T1[schemaOnce - Tier 1]
    MC[MetadataCache - Tier 2]
    SC[statCache - Tier 3]
    PC[pathCache]
    UC[undoCache]
  end

  PG[(PostgreSQL)]

  FUSE --> ops
  NFS --> ops
  T1 --> PG
  MC --> PG
  SC -.->|Stat only| PG
  PC -.->|resolve_path| PG
  UC -.->|preview only| PG
  ops -->|ReadFile ReadDir exports| PG
```

### Tier 1: Schema resolution (`sync.Once`)

`resolveSchema` fills empty `FSContext.Schema` from `current_schema()` once per mount via `schemaOnce`. This removes repeated `SELECT current_schema()` on every RPC (roughly 4–6 queries per operation before caching).

- **Invalidation:** None until mount restart.
- **Scope:** Default-schema paths such as `/users` (not explicit `/.schemas/foo/...` paths).

### Tier 2: `MetadataCache`

`MetadataCache` (`internal/tigerfs/fs/metadata_cache.go`) caches information_schema–style data in two TTL sub-tiers:

| Sub-tier | Config key | Default | Contents |
|----------|------------|---------|----------|
| Catalog | `metadata_refresh_interval` | `10s` | Schemas, table lists, view lists per schema |
| Structural | `structural_metadata_refresh_interval` | `5m` | Row count estimates (`pg_class.reltuples`), batch permissions |
| Per-table (no TTL) | — | Until invalidate | Column lists, primary keys (positive and negative cache) |

`Invalidate()` clears all tiers, PK/column maps, and per-schema lazy caches. Call sites include DDL commit (`write.go`), and synth `.build/` / `.format/` (`build.go`).

<ParamField body="metadata_refresh_interval" type="duration">
Catalog refresh interval. Env: `TIGERFS_METADATA_REFRESH_INTERVAL`. YAML under metadata section in config display.
</ParamField>

<ParamField body="structural_metadata_refresh_interval" type="duration">
Structural refresh interval. Env: `TIGERFS_STRUCTURAL_METADATA_REFRESH_INTERVAL`.
</ParamField>

### Tier 3: `statCache` (ReadDir-primed stat metadata)

`statCache` stores `Entry` metadata (name, `IsDir`, `Size`, `ModTime`, `Mode`) keyed by `schema + "\x00" + table`, with per-filename entries inside a `tableCache` bucket.

| Behavior | Detail |
|----------|--------|
| TTL | `statCacheTTL` = **2 seconds** (table bucket expires entirely) |
| Prime | `ReadDir` on tables, synth views, and row directories calls `statCache.prime` |
| Lookup | `Stat` checks cache before PostgreSQL (native rows, synth files, pipeline info/export **sizes**) |
| Negative cache | Missing synth/native files can be cached as negatives for up to 2s |
| Row columns | Keys use `schema\x00table/rowPK` prefix; `invalidate` clears table and all `table/` row-level buckets |

`ReadFile` and `readExportFile` **do not** use `statCache` for content—they always query PostgreSQL. Stat may cache the **size** of an export by running `readExportFile` once inside `statExport`, but a subsequent `cat` still fetches fresh data.

### `pathCache` (synth hierarchy)

For parent-pointer synth directories (ADR-017), `pathCache` maps `(parentID, filename) → row UUID` per table, with the same **2s** TTL as stat cache (`pathCacheTTL` in `path_cache.go`). Invalidated alongside `statCache` on directory structure changes (rename, move, delete, undo).

### `undoCache` (preview only)

Undo preview paths (`.undo/.../.info/summary`, affected-file listings) use a **2s** TTL cache over read-only preview queries. **Apply** (`.apply`) re-queries inside `ExecuteUndoTransaction` without using this cache. A narrow stale-preview window is possible if apply commits while a preview `SELECT` is in flight; apply correctness does not depend on cache freshness.

## Cross-mount read freshness

### Content reads (always fresh)

```text
Mount A: echo "new" > /mnt/a/app/note.md
Mount B: cat /mnt/b/app/note.md     → queries DB, sees "new" immediately
```

`readFileWithParsed` routes row/column/export/history reads to `db.GetRow`, `db.GetColumn`, pipeline export queries, or synth synthesis—none consult a content cache.

### Directory listings

`ReadDir` always lists from PostgreSQL (or pipeline SQL). It may **prime** `statCache` as a side effect, but the listing itself is not served from cache.

### Stat and `ls -l` staleness

NFS and FUSE issue many `Stat`/`GETATTR` calls per directory entry. Without caching, synth `ls -l` could trigger tens of full-table scans; with ReadDir-primed stat cache, one ReadDir plus cached Stats suffices (ADR-014).

Cross-mount visibility for **metadata**:

- Another mount’s write calls `statCache.invalidate(userSchema, table)` → immediate on that process.
- If invalidation is missed (wrong schema key—see below), negatives can survive up to **2s**.
- Another mount with no write: stat entries expire after **2s** TTL.

<Info>
macOS mounts use NFS `noac` so the **kernel** does not cache attributes for ~60s. TigerFS still serves GETATTR from its in-memory stat cache (2s). `noac` adds loopback round-trips but does not multiply SQL load.
</Info>

### FUSE kernel caches (separate layer)

Linux FUSE exposes tunable kernel attribute caching:

| Flag | Default | Role |
|------|---------|------|
| `attr_timeout` | `1s` | FUSE attribute cache |
| `entry_timeout` | `1s` | FUSE dentry cache |

These are independent of TigerFS’s `statCache`. They affect how long the **kernel** may reuse inode attributes, not whether TigerFS queries PostgreSQL for content.

## Write invalidation

Successful writes must drop cached metadata for the affected table so other mounts and subsequent Stats do not serve stale directory structure or false negatives.

### `statCache` and `pathCache`

`WriteFile` documents postconditions: stat/path caches for the table are invalidated (`write.go`). Typical call pattern:

```go
o.statCache.invalidate(fsCtx.Schema, fsCtx.TableName)
o.pathCache.invalidate(fsCtx.Schema, fsCtx.TableName)  // synth hierarchy ops
```

Also invalidated on: synth create/update/delete/rename, undo apply, and related synth ops (`synth_ops.go`, `undo.go`).

<Warning>
Pass the **user-facing schema** (the view/workspace schema), not `synth.TigerFSSchema`. Undo and synth paths resolve stats against the user schema; invalidating `tigerfs` leaves stale negatives on the workspace mount for up to 2s. `undo.go` documents this explicitly for apply.
</Warning>

### `MetadataCache`

| Trigger | Action |
|---------|--------|
| DDL `.commit` | `metaCache.Invalidate()` |
| `.build/` / `.format/` synth setup | `invalidateSynthCache()` + `metaCache.Invalidate()` |
| PK DDL | `InvalidatePrimaryKey(schema, table)` |
| TTL expiry | Automatic catalog/structural refresh on next access |

### `undoCache`

Cleared on undo apply (`undoCache.invalidate()`), together with stat/path invalidation under the user schema.

## Pipeline cache key isolation

Pipeline paths build SQL from `FSContext` (filters, order, limits, column projection). **Stat** for pipeline-scoped virtual files must not share cache entries with unfiltered paths on the same table.

TigerFS uses the **full mount path string** as the stat cache filename key for:

- **`.info/` files** — e.g. `.filter/active/true/.info/count` vs `.info/count` (`statInfoFile` / `cachedStat`)
- **`.export/<format>` files** — e.g. `.filter/in_stock/true/.export/json` vs `.export/json` (`statExport`)

Integration tests in `test/integration/command_cases.go` (`ExportAllProductsJSON` → filtered export → `ExportAllProductsJSON_AfterFiltered`) verify that a filtered export stat does not corrupt the unfiltered export size used by NFS `READ`.

<Check>
`cat` on any export path always runs `readExportFile` against live pipeline state. Stat cache only affects reported **size** for GETATTR; read content is unaffected.
</Check>

Table-level stat cache keys remain `schema\x00table` for row directories and synth filenames; pipeline state is not part of the table key—only paths that encode pipeline in the **filename key** (info/export) get isolation.

## What stays uncached or lightly cached

| Operation | Behavior |
|-----------|----------|
| `ReadFile` / `cat` | DB query every time |
| `ReadDir` / `ls` | DB (or pipeline) query; may prime stat cache |
| `readExportFile` | Pipeline-aware SQL every time |
| `readColumnFile` | `GetColumn` every time |
| Undo `.apply` | Fresh transaction, no undo preview cache |
| Synth targeted lookups | `RowExistsByColumns` / `GetRowByColumns` on stat miss (ADR-014) |

## Configuration reference

Defaults from `config.Init()` (`internal/tigerfs/config/config.go`):

```yaml
metadata_refresh_interval: 10s
structural_metadata_refresh_interval: 5m
attr_timeout: 1s      # FUSE only
entry_timeout: 1s     # FUSE only
```

Precedence: defaults → `~/.config/tigerfs/config.yaml` → `TIGERFS_*` env → CLI flags (see [Configuration reference](/configuration-reference)).

## Operational expectations

| Scenario | Expected behavior |
|----------|-------------------|
| Cross-mount `cat` after write | Immediate (no content cache) |
| Cross-mount `ls` after external INSERT | New row may appear up to ~2s late on stat-primed listings; direct `cat` on new path works immediately |
| Cross-mount DDL (new table) | Visible after catalog TTL (default 10s) or `metaCache.Invalidate()` from a TigerFS DDL commit on any mount |
| `ls -l` on synth dir | Fast after one ReadDir; Stats hit stat cache |
| Undo restore, `ls` on macOS NFS | `noac` + TigerFS invalidation; restored rows get fresh `modified_at` for client readdir |
| False `ENOENT` for existing file | Check debug logs for `statCache.isNegative hit`; suspect stale negative or wrong-schema invalidation |

## Debugging

Enable debug logging on the mount (`--debug`) and look for:

- `statCache.lookup hit` / `statCache.isNegative hit`
- `statCache.invalidate` with `schema` and `table`
- `Metadata cache catalog expired, refreshing`

Stress runs may log NFS monotonicity warnings on `.log/.last/N/.export/json`; those reflect macOS NFS client staleness, not TigerFS content caching (see project stress-test notes in `CLAUDE.md`).

## Related pages

<CardGroup>
  <Card title="Capability directories" href="/capability-directories">
    Pipeline path grammar and how FSContext drives SQL for exports and listings.
  </Card>
  <Card title="Platform backends" href="/platform-backends">
    FUSE vs NFS adapters, noac, and NFS write/stat interaction with stat cache.
  </Card>
  <Card title="Synthesized apps" href="/page-synthesized-apps">
    Synth ReadDir priming, pathCache, and targeted stat queries.
  </Card>
  <Card title="History, savepoints, and undo" href="/history-savepoints-undo">
    Undo preview cache, apply path freshness, and post-undo invalidation.
  </Card>
  <Card title="Troubleshooting" href="/troubleshooting">
    Stale mounts, debug logging, and metadata refresh recovery.
  </Card>
  <Card title="Configuration reference" href="/configuration-reference">
    Full config keys including metadata and FUSE timeout fields.
  </Card>
</CardGroup>