# Boot from existing Postgres data

> Constraints for reusing a Postgres 18.3 PGDATA: control file checks, major-version expectations, and practical recovery when byte order or catalog state does not match.

- Repository: malisper/pgrust
- GitHub: https://github.com/malisper/pgrust
- Human docs: https://grok-wiki.com/public/docs/malisper-pgrust-7e0c8e82bc21
- Complete Markdown: https://grok-wiki.com/public/docs/malisper-pgrust-7e0c8e82bc21/llms-full.txt

## Source Files

- `README.md`
- `crates/common/controldata_utils/src/lib.rs`
- `crates/backend/postmaster/postmaster/src/main_entry.rs`
- `crates/backend/main/initdb/src/lib.rs`
- `scripts/run-recovery-tap`

---

---
title: "Boot from existing Postgres data"
description: "Constraints for reusing a Postgres 18.3 PGDATA: control file checks, major-version expectations, and practical recovery when byte order or catalog state does not match."
---

pgrust claims on-disk compatibility with PostgreSQL 18.3: the same `postgres` binary can start postmaster against a data directory created by C PostgreSQL 18.3 or by `postgres --initdb`, as long as `PG_VERSION`, `global/pg_control`, and the compile-time layout constants match what this server was built for. Boot does not rewrite an incompatible cluster; validation is fail-closed.

<Warning>
pgrust is not production-ready. Reusing a real Postgres 18.3 data directory is for development, recovery experiments, and regression-style verification—not a supported production cutover path.
</Warning>

## Compatibility contract

| Surface | Expected value / behavior |
| --- | --- |
| Major version in `PGDATA/PG_VERSION` | Major `18` (initdb writes `18`; server compares major of `18.3`) |
| Control file | `PGDATA/global/pg_control` present, fixed-layout image |
| `PG_CONTROL_VERSION` | `1800` |
| `CATALOG_VERSION_NO` | `202506291` (PostgreSQL 18.3) |
| Tablespace version dir name | `PG_18_202506291` |
| Host endian / struct layout | Native-endian decode of control fields; wrong endian fails control checks |
| Extensions / PLs | Generic C extensions and PL/Python, PL/Perl, PL/Tcl are not generally usable even if the on-disk cluster is compatible |

There is no `pg_upgrade` workflow in this repository. Wrong major, wrong catalog version, wrong byte order, or mismatched compile-time sizes require a cluster that already matches 18.3 layout, or a fresh `--initdb`.

## Prerequisites

- Built `target/release/postgres` (or image `malisper/pgrust`) with share path wiring that can still serve config samples if you re-init; for boot-only of an existing tree, share path matters less than for initdb, but runtime `-c` settings still apply.
- A PostgreSQL **18.x** data directory (major `18`), ideally from **18.3**, cleanly shut down when possible.
- Same process user as the directory owner (ownership check is fatal).
- Directory mode without group-write or world bits (allowed modes effectively `0700` / `0750`).
- Mandatory runtime GUCs when starting pgrust: `io_method=sync`, adequate stack (`ulimit -s`, `RUST_MIN_STACK`, `max_stack_depth`).

## Boot sequence against existing `PGDATA`

Postmaster startup validates the data directory before accepting connections:

```text
SelectConfigFiles (-D / config)
        │
        v
 checkDataDir ──► exists, is dir, owner=geteuid, mode ok
        │
        v
 ValidatePgVersion ──► major from PG_VERSION == major of "18.3"
        │
        v
 check_control_file ──► open PGDATA/global/pg_control (existence only)
        │
        v
 ChangeToDataDir + CreateDataDirLockFile
        │
        v
 LocalProcessControlFile → ReadControlFile
        │                 (version, CRC, catalog, layout fields, WAL seg size)
        v
 continue postmaster init (preload, shmem, …)
```

### Existence-only control check

Before full parse, postmaster confirms `global/pg_control` is openable. Missing file exits with a log-style diagnostic and postmaster exit code `2`:

```text
could not find the database system: expected to find pg_control at "<DataDir>/global/pg_control"
```

Full CRC and compatibility validation happens in `ReadControlFile`, not in this existence probe.

## `PG_VERSION` major check

`ValidatePgVersion` reads the first whitespace-delimited token from `PGDATA/PG_VERSION` (max 63 characters) and compares its leading integer major to the server’s major from `PG_VERSION` (`"18.3"` → major `18`).

| Condition | Outcome |
| --- | --- |
| Missing `PG_VERSION` | FATAL: `"… is not a valid data directory"` |
| Empty / non-numeric token | Same invalid-directory FATAL |
| Major ≠ server major | FATAL: `database files are incompatible with server` |
| Major matches (`18`) | Proceed |

pgrust initdb writes top-level `PG_VERSION` as `18` (major only), matching C initdb major-file semantics. Minor differences within major 18 are not rejected by this check; catalog and control fields still must match.

## Control file layout and readers

On-disk `global/pg_control` is zero-padded to `PG_CONTROL_FILE_SIZE` (8192). Readers take the leading `ControlFileData` image (`SIZE_OF_CONTROL_FILE_DATA` = 296 bytes on LP64, CRC at offset 292). Fields are decoded with **native endian** integers.

Two readers share the image but differ in severity:

| Reader | Used for | CRC mismatch | `PG_CONTROL_VERSION` / byte order |
| --- | --- | --- | --- |
| `get_controlfile` (`controldata_utils`) | SQL `pg_control_*` helpers | Returns `crc_ok=false` (caller may error) | Hard ERROR; byte-order heuristic then exact version |
| `ReadControlFile` (startup via `LocalProcessControlFile`) | Postmaster / backend boot | Hard error: `incorrect checksum in control file` | Hard error: incompatible control version |

### Byte-order heuristic (`get_controlfile`)

If `pg_control_version % 65536 == 0` and `pg_control_version / 65536 != 0`, the server treats the image as **wrong endian** for this host:

```text
byte ordering mismatch in control file "<DataDir>/global/pg_control"
detail: Possibly byte ordering doesn't match the database server that wrote this control file.
```

Otherwise, if `pg_control_version != PG_CONTROL_VERSION` (`1800`):

```text
the database cluster was initialized with PG_CONTROL_VERSION … but the server was compiled with PG_CONTROL_VERSION 1800 (0x00000708)
hint: This could be a problem of mismatched byte ordering.  It looks like you need to initdb.
```

### Startup compatibility fields (`ReadControlFile`)

After requiring `pg_control_version == 1800` and a matching CRC, startup equality-checks compile-time layout constants:

| Control field | Server expectation |
| --- | --- |
| `catalog_version_no` | `202506291` |
| `maxAlign` | `8` |
| `floatFormat` | `1234567.0` (`FLOATFORMAT_VALUE`) |
| `blcksz` | `8192` |
| `relseg_size` | `131072` |
| `xlog_blcksz` | matches `XLOG_BLCKSZ` |
| `nameDataLen` | `64` |
| `indexMaxKeys` | `32` |
| `toast_max_chunk_size` | `1996` |
| `loblksize` | `2048` |
| `float8ByVal` | `true` |
| `xlog_seg_size` | Valid WAL segment size (power-of-two range as in Postgres) |

Mismatch shape:

```text
database files are incompatible with server: <NAME> <on-disk> but server <expected>
```

Special cases: float format → `float format mismatch`; float8 by-value → `USE_FLOAT8_BYVAL mismatch`; bad WAL segment size → `invalid WAL segment size in control file (… bytes)`.

## Start an existing data directory

Same command shape as a pgrust-created cluster; only `-D` points at the foreign tree.

```bash
ulimit -s 65520

RUST_MIN_STACK=33554432 target/release/postgres \
  -D /path/to/pg18-data \
  -F \
  -c listen_addresses= \
  -k /tmp \
  -p 5432 \
  -c io_method=sync \
  -c max_stack_depth=60000
```

Verify:

```bash
psql -h /tmp -p 5432 -U postgres -d postgres \
  -c "select version(), current_setting('server_version_num')"
```

`version()` is branded `pgrust 18.3 …`; `server_version_num` remains `180003`. A successful connect after the control checks is the readiness signal for an existing tree.

### Docker volume reuse

The Docker entrypoint treats a non-empty `PG_VERSION` under `PGDATA` as `DATABASE_ALREADY_EXISTS` and skips `--initdb`. Point the volume at a compatible 18.x tree and start the image as usual; first-boot init scripts under `/docker-entrypoint-initdb.d` run only when the directory was empty.

## Practical recovery when validation fails

pgrust does **not** auto-repair control files, byte-swap catalogs, or upgrade majors. Recovery is operational:

| Failure | Practical action |
| --- | --- |
| Wrong major (`PG_VERSION` not 18) | Use a true PG 18 cluster, or create a new one with `postgres --initdb` |
| Byte-order / wrong-endian control file | Run pgrust on a host with the **same endianness** as the writer, or rebuild data on this host |
| `PG_CONTROL_VERSION` ≠ 1800 | Cluster is not PG 18 control format; re-init or use a matching 18.x cluster |
| `CATALOG_VERSION_NO` ≠ `202506291` | Catalog is not the 18.3 catalog this binary expects; re-init with this binary or with matching C 18.3 |
| Layout field mismatch (BLCKSZ, MAXALIGN, …) | Binary and data were not built with the same compile-time sizes; use matching builds or re-init |
| Incorrect checksum | Corrupt or partially written control file; restore from backup or re-init |
| Missing `global/pg_control` | Incomplete data directory; restore or re-init |
| `directory … exists but is not empty` from `--initdb` | initdb refuses non-empty targets; use a new path or remove/rename the old tree after backup |

### Fresh cluster when reuse is impossible

```bash
target/release/postgres --initdb \
  -D /tmp/pgrust-data-new \
  -L "$PWD/vendor/postgres-18.3/share" \
  --no-locale \
  --encoding UTF8 \
  -U postgres
```

Success ends with:

```text
Success. You can now start the database server using pgrust/postgres -D …
```

Do not run `--initdb` into a live production path without an explicit backup and empty (or new) directory.

## Cross-engine verification (recovery TAP)

`scripts/run-recovery-tap` is the in-repo proof pattern that a **C-initdb’d** cluster can be driven by the **pgrust** postmaster: the harness fakes an install where `initdb` / `pg_ctl` / client tools come from a C PostgreSQL install, while `postgres` is a real copy of the pgrust binary. It injects:

```text
io_method = sync
max_stack_depth = 7000kB
```

via `TEMP_CONFIG` so TAP nodes start under pgrust constraints. That path exercises archive recovery and related suites against on-disk state produced by C tools—not a guarantee for every production extension or custom build option.

## Constraints that still apply after a successful control check

- **IO method**: default worker IO is unported; without `io_method=sync` the server may fail early.
- **Stack**: large frames need raised `ulimit -s`, `RUST_MIN_STACK`, and `max_stack_depth`.
- **Auth / extensions**: Docker notes that password host auth can still fail until password seams are complete; unported extensions will not load even if the cluster catalogs reference them.
- **Ownership / permissions**: wrong owner or overly open directory modes fail before control parse.
- **Lock file**: only one postmaster per data directory; existing `postmaster.pid` from a crashed host may need the usual Postgres cleanup after confirming no live process.

## Failure signals (quick map)

| Symptom | Likely stage |
| --- | --- |
| `data directory "…" does not exist` / not a directory | `checkDataDir` |
| `has wrong ownership` / `invalid permissions` | `checkDataDir` |
| `is not a valid data directory` | Missing or unreadable `PG_VERSION` |
| `database files are incompatible with server` (no field name) | Major version mismatch |
| `could not find the database system: expected to find pg_control` | Existence probe |
| `byte ordering mismatch` / `need to initdb` | Control version / endian |
| `incorrect checksum in control file` | CRC at startup |
| `CATALOG_VERSION_NO … but server 202506291` | Catalog epoch mismatch |
| Panic / early exit mentioning `io_method` or stack | Runtime GUC / process limits after control accepted |

## Next

<CardGroup>
  <Card title="Initialize a cluster" href="/init-cluster">
    Create a matching PGDATA with postgres --initdb when reuse is not possible.
  </Card>
  <Card title="Run the server" href="/run-server">
    Required -D, socket/TCP, io_method=sync, and stack settings for a successful postmaster start.
  </Card>
  <Card title="Compatibility and status" href="/compatibility">
    On-disk compatibility claims, non-production status, and extension boundaries.
  </Card>
  <Card title="Troubleshooting" href="/troubleshooting">
    Known failure modes including stack, io_method, share path, and root refusal.
  </Card>
  <Card title="Test harness reference" href="/test-harness-reference">
    Environment for recovery TAP and other harnesses that mix C tools with pgrust.
  </Card>
</CardGroup>