# Data directories

> On-disk layout for OpenClaw (~/.openclaw/memory-tdai), offload (~/.openclaw/context-offload), and Hermes MEMORY_TENCENTDB_ROOT trees: conversations, records, scene_blocks, vectors.db, persona, checkpoints.

- Repository: TencentCloud/TencentDB-Agent-Memory
- GitHub: https://github.com/TencentCloud/TencentDB-Agent-Memory
- Human docs: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97
- Complete Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/llms-full.txt

## Source Files

- `index.ts`
- `src/utils/pipeline-factory.ts`
- `src/utils/manifest.ts`
- `src/utils/checkpoint.ts`
- `src/utils/openclaw-state-dir.ts`
- `scripts/README.memory-tencentdb-ctl.md`
- `src/offload/storage.ts`

---

---
title: "Data directories"
description: "On-disk layout for OpenClaw (~/.openclaw/memory-tdai), offload (~/.openclaw/context-offload), and Hermes MEMORY_TENCENTDB_ROOT trees: conversations, records, scene_blocks, vectors.db, persona, checkpoints."
---

TencentDB Agent Memory persists long-term memory and short-term context offload as separate directory trees. The OpenClaw plugin hard-codes the data folder name `memory-tdai` under the OpenClaw state dir (typically `~/.openclaw/memory-tdai`). Context offload defaults to `~/.openclaw/context-offload`. Standalone Gateway / Hermes mode defaults under `$MEMORY_TENCENTDB_ROOT` (typically `~/.memory-tencentdb/memory-tdai`). Package renames do **not** rename the on-disk data folder.

## Root paths

| Surface | Default root | Override | Owns |
| --- | --- | --- | --- |
| OpenClaw long-term memory | `~/.openclaw/memory-tdai` | `OPENCLAW_STATE_DIR` (or host `runtime.state.resolveStateDir()`) + fixed `memory-tdai` suffix | L0–L3 JSON/Markdown, SQLite `vectors.db`, checkpoints |
| Context offload | `~/.openclaw/context-offload` | `offload.dataDir` (absolute path) | Per-agent tool-pair offload, refs, Mermaid MMDs, `state.json` |
| Hermes / standalone Gateway | `~/.memory-tencentdb/memory-tdai` | `TDAI_DATA_DIR` or `MEMORY_TENCENTDB_ROOT` | Same memory tree as OpenClaw, plus `tdai-gateway.json` / logs |
| Hermes install code | `~/.memory-tencentdb/tdai-memory-openclaw-plugin` | `TDAI_INSTALL_DIR` | Gateway source + `node_modules` (not memory content) |

OpenClaw state-dir resolution order:

1. Host `runtime.state.resolveStateDir()`
2. `OPENCLAW_STATE_DIR`
3. `~/.openclaw`

Plugin data dir is always `path.join(stateDir, "memory-tdai")`.

Gateway / Hermes data-dir resolution order:

1. `TDAI_DATA_DIR` (or config `data.baseDir`, with `~/` expansion)
2. `$MEMORY_TENCENTDB_ROOT/memory-tdai` (default root `~/.memory-tencentdb`)
3. Legacy fallback `~/memory-tdai` when the new default does not exist yet

```text
OpenClaw host                         Hermes / standalone Gateway
─────────────────                     ───────────────────────────
$OPENCLAW_STATE_DIR                   $MEMORY_TENCENTDB_ROOT
  └── memory-tdai/   (L0–L3)            ├── memory-tdai/          ($TDAI_DATA_DIR)
  └── context-offload/ (optional)       │     ├── conversations/…
                                        │     └── tdai-gateway.json
                                        └── tdai-memory-openclaw-plugin/  (install)
```

<Note>
The npm package may be `@tencentdb-agent-memory/memory-tencentdb`, but the OpenClaw data directory remains `memory-tdai`. Uninstalling the plugin does not delete that tree.
</Note>

## Memory data tree (`memory-tdai`)

`initDataDirectories()` creates the core subdirectories on plugin (or Gateway) start:

:::files
~/.openclaw/memory-tdai/          # or $TDAI_DATA_DIR
├── conversations/                # L0 daily JSONL shards
│   └── YYYY-MM-DD.jsonl
├── records/                      # L1 daily JSONL shards
│   └── YYYY-MM-DD.jsonl
├── scene_blocks/                 # L2 scene Markdown files
│   └── *.md
├── persona.md                    # L3 persona (root file)
├── vectors.db                    # SQLite backend only (sqlite-vec + FTS5)
├── .metadata/
│   ├── manifest.json             # store binding (sqlite|tcvdb)
│   ├── recall_checkpoint.json    # L0/L1/L2/L3 pipeline cursors
│   ├── scene_index.json          # engineering index of scene_blocks
│   └── instance_id               # stable UUID for metrics
└── .backup/
    ├── persona/                  # timestamped persona.md copies
    └── scene_blocks/             # timestamped directory snapshots
:::

| Path | Layer | Format | Notes |
| --- | --- | --- | --- |
| `conversations/YYYY-MM-DD.jsonl` | L0 | One message per line | All sessions share the daily shard; `sessionKey` is a field, not the filename |
| `records/YYYY-MM-DD.jsonl` | L1 | One memory atom per line | Append-only; updates/merges append new lines and delete from the vector store |
| `scene_blocks/*.md` | L2 | Scene Markdown | LLM workspace is sandboxed to this directory during extraction |
| `persona.md` | L3 | Markdown | Written by persona generation; also receives scene navigation updates |
| `vectors.db` | L1 index | SQLite | Present when `storeBackend` is `sqlite` (default). Remote `tcvdb` does not create this file |
| `.metadata/*` | control plane | JSON / plain text | Manifest, checkpoint, scene index, instance id |
| `.backup/*` | recovery | files / dirs | Pre-LLM snapshots for persona and scene blocks |

### Layer file shapes

**L0 (`conversations/`)** — incremental capture from `agent_end`. Daily shard name uses the configured local timezone date. Each line is a flat message record (`sessionKey`, `sessionId`, `recordedAt`, `id`, `role`, `content`, `timestamp`).

**L1 (`records/`)** — extraction output, also daily-sharded. JSONL remains the durable local log; hybrid search/dedup prefers `vectors.db` or Tencent VectorDB when the store is healthy, and falls back to scanning JSONL when the store is degraded or missing.

**L2 (`scene_blocks/`)** — one `.md` file per scene. `scene_index.json` under `.metadata/` is maintained by the engineering side (not by the sandboxed LLM). Filenames are normalized after extraction so navigation links stay stable.

**L3 (`persona.md`)** — single file at the data-dir root. Before regeneration, the previous file is copied into `.backup/persona/` with a timestamped name.

### Store backend and `vectors.db`

| `storeBackend` | Local index file | Remote data |
| --- | --- | --- |
| `sqlite` (default) | `<dataDir>/vectors.db` | None |
| `tcvdb` | No local vector DB file | Collections in the configured Tencent VectorDB database |

`.metadata/manifest.json` records the first successful store binding:

- SQLite: `{ "type": "sqlite", "sqlite": { "path": "vectors.db" } }`
- TCVDB: `{ "type": "tcvdb", "tcvdb": { "url", "database", "alias?" } }`

Later config drift is logged at debug level; the manifest is not overwritten on every start. Seed runs may also set the `seed` field once.

### Checkpoints and metadata

| File | Role |
| --- | --- |
| `.metadata/recall_checkpoint.json` | Split-state checkpoint: global counters plus per-session `runner_states` (L0 capture cursor, L1 cursor, last scene name) and `pipeline_states` (conversation counts, extraction times, warm-up threshold) |
| `.metadata/scene_index.json` | Array of `{ filename, summary, heat, created, updated }` for fast L2 lookup |
| `.metadata/instance_id` | UUID created once; used in metric log lines |
| `.metadata/manifest.json` | Store binding + optional seed summary |

Checkpoint writes use temp-file + rename under a per-file async lock so concurrent `agent_end` / pipeline steps do not corrupt the file.

### Backups

`BackupManager` stores copies under `.backup/<category>/`:

| Category | When | Name pattern | Retention knobs |
| --- | --- | --- | --- |
| `persona` | Before L3 LLM write | `persona_<YYYYMMDD_HHmmss>_offsetN.md` | `persona.backupCount` (default 3) |
| `scene_blocks` | Before L2 LLM extraction | `scene_blocks_<YYYYMMDD_HHmmss>_offsetN/` (shallow file copy) | `persona.sceneBackupCount` |

If L2 extraction fails, the latest `scene_blocks` backup can be restored over the live directory.

### Local JSONL retention

Optional cleaner (`memoryCleanup.retentionDays` + `cleanTime`, default clean time `03:00`) deletes expired **daily shards** under `conversations/` and `records/` only. It does not remove `scene_blocks/`, `persona.md`, `vectors.db`, or `.metadata/`. Safety floors keep at least ~50 L0 and ~20 L1 records when counting would otherwise wipe the store.

## Context offload tree

Offload is independent of `memory-tdai`. Default root: `~/.openclaw/context-offload`, override with plugin config `offload.dataDir`.

:::files
~/.openclaw/context-offload/          # dataRoot
└── <agentName>/                      # one dir per agent (from sessionKey)
    ├── offload-<sessionId>.jsonl     # per-session tool-pair offload log
    ├── state.json                    # agent/session offload state
    ├── sessions-registry.json        # sessionKey → real sessionId map
    ├── refs/                         # full tool results as Markdown
    │   └── <sanitized-iso-timestamp>.md
    ├── mmds/                         # Mermaid canvas files
    │   └── *.mmd
    └── skills/                       # optional skill artifacts from L4
        └── <skillName>/
:::

### Isolation rules

| Scope | Shared | Isolated |
| --- | --- | --- |
| Different agents | Separate under `dataRoot` | Entire agent directory |
| Same agent, different sessions | `mmds/`, `refs/`, `state.json` | `offload-<sessionId>.jsonl` |
| Multi-worker `swebench-wN` session keys | — | Worker suffix folded into `agentName` so each worker gets its own `dataDir` |

`sessionKey` format expected by offload: `agent:<agent-name>:<session-id>`. Path components are sanitized (unsafe characters → `_`).

### Key offload files

| Path | Purpose |
| --- | --- |
| `offload-*.jsonl` | Append-only offload entries (tool pairs, node ids, L1.5/L2 linkage) |
| `refs/*.md` | Full tool-result bodies; header includes tool name + timestamp |
| `mmds/*.mmd` | Symbolic Mermaid canvas used for injection |
| `state.json` | Persistent offload state for the agent directory |
| `sessions-registry.json` | Maps OpenClaw `sessionKey` to the real session id and offload filename |

Optional reclaimer (`offload.offloadRetentionDays`, minimum effective value 3) can expire old session JSONL files, orphan refs, and non-active MMDs under this tree. Debug logs under `dataRoot` can be size-capped via `offload.logMaxSizeMb` (default 50).

## Hermes and Gateway paths

Standalone / Hermes deployments consolidate under `MEMORY_TENCENTDB_ROOT` (default `~/.memory-tencentdb`):

:::files
~/.memory-tencentdb/
├── memory-tdai/                          # $TDAI_DATA_DIR — same L0–L3 layout
│   ├── conversations/
│   ├── records/
│   ├── scene_blocks/
│   ├── persona.md
│   ├── vectors.db                        # if sqlite
│   ├── .metadata/
│   ├── .backup/
│   ├── tdai-gateway.json                 # or tdai-gateway.yaml (mode 0600 typical)
│   └── logs/                             # standalone: gateway.stdout/stderr.log, gateway.pid
└── tdai-memory-openclaw-plugin/          # $TDAI_INSTALL_DIR — plugin source + node_modules
    └── src/gateway/server.ts
:::

Additional Hermes-mode paths (outside the memory tree):

| Path | Mode | Role |
| --- | --- | --- |
| `$HERMES_HOME/logs/memory_tencentdb/` | hermes | Gateway stdout/stderr when supervised by Hermes |
| `$HERMES_HOME/env.d/memory-tencentdb-llm.sh` | hermes | LLM credentials for Gateway child process |
| `/etc/profile.d/memory-tencentdb-env.sh` | install | `MEMORY_TENCENTDB_GATEWAY_CMD` and related env |

Gateway config file search order:

1. `TDAI_GATEWAY_CONFIG`
2. `./tdai-gateway.yaml` or `./tdai-gateway.json` in CWD
3. `$TDAI_DATA_DIR/tdai-gateway.yaml` or `tdai-gateway.json`
4. Environment-only config

Legacy pre-0.4 locations `~/memory-tdai` and `~/tdai-memory-openclaw-plugin` are still detected; install scripts can migrate them into `$MEMORY_TENCENTDB_ROOT`.

## Seed output directories

`openclaw memory-tdai seed` writes a **separate** data tree (does not mutate the live plugin dir by default):

| Option | Result |
| --- | --- |
| `--output-dir <path>` | Explicit absolute/relative path |
| Default | `<stateDir>/memory-tdai-seed-<YYYYMMDD-HHmmss>` |

The seed runtime reuses `initDataDirectories` + the same L0/L1/L2/L3 writers, so the output tree matches the live layout (`conversations/`, `records/`, `scene_blocks/`, `persona.md`, `vectors.db`, `.metadata/`). Promote seed results into the live dir only with an explicit operational copy/merge after verification.

## Environment and config overrides

| Variable / config | Affects | Default |
| --- | --- | --- |
| `OPENCLAW_STATE_DIR` | OpenClaw state parent | `~/.openclaw` |
| (implicit) `memory-tdai` segment | Memory data folder name | hard-coded |
| `offload.dataDir` | Offload root | `~/.openclaw/context-offload` |
| `MEMORY_TENCENTDB_ROOT` | Hermes/standalone root | `~/.memory-tencentdb` |
| `TDAI_DATA_DIR` | Gateway data dir | `$MEMORY_TENCENTDB_ROOT/memory-tdai` |
| `TDAI_INSTALL_DIR` | Gateway install/code dir | `$MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin` |
| `TDAI_GATEWAY_CONFIG` | Gateway config file path | searched as above |
| `MEMORY_TENCENTDB_LOG_DIR` | Gateway log dir override | mode-dependent |
| `HERMES_HOME` | Hermes home | `~/.hermes` |

## Verify on disk

After a successful OpenClaw enable + gateway restart:

```bash
# Long-term memory
ls -la "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/"
ls "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/conversations/" 2>/dev/null
ls "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/records/" 2>/dev/null
test -f "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/.metadata/recall_checkpoint.json" && echo "checkpoint ok"
test -f "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/vectors.db" && echo "sqlite vectors present" || echo "no local vectors.db (tcvdb or not yet indexed)"

# Context offload (if enabled)
ls -la "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/context-offload/" 2>/dev/null

# Hermes / standalone
ls -la "${TDAI_DATA_DIR:-$HOME/.memory-tencentdb/memory-tdai}/"
```

Healthy signals:

- Subdirs `conversations`, `records`, `scene_blocks`, `.metadata`, `.backup` exist after first init
- L0 files appear after conversation capture; L1/L2/L3 appear after pipeline runs
- Gateway health does not require non-empty L1/L2; empty trees after fresh install are normal
- After package rename, the same `~/.openclaw/memory-tdai` tree continues to grow

## Related pages

<CardGroup>
  <Card title="Memory layers" href="/memory-layers">
    L0–L3 producers, storage shapes, and pipeline triggers that fill this tree.
  </Card>
  <Card title="Context offload" href="/context-offload">
    How tool-pair offload, Mermaid injection, and node_id drill-down use the offload tree.
  </Card>
  <Card title="Configure storage backends" href="/configure-storage">
    sqlite vs tcvdb, embedding settings, and store health checks tied to vectors.db / remote DB.
  </Card>
  <Card title="Seed historical conversations" href="/seed-conversations">
    Seed CLI flags and verification of a separate seed output directory.
  </Card>
  <Card title="Gateway control" href="/gateway-control">
    memory-tencentdb-ctl paths, log locations, and TDAI_* env overrides.
  </Card>
  <Card title="Export diagnostics" href="/export-diagnostics">
    Packages redacted config, logs, and the memory-tdai tree for support.
  </Card>
  <Card title="Package rename migration" href="/package-rename">
    Preserve ~/.openclaw/memory-tdai when moving from @tdai/memory-tdai to the new package name.
  </Card>
</CardGroup>