# TencentDB Agent Memory Documentation
> Source-backed docs for the @tencentdb-agent-memory/memory-tencentdb four-layer memory plugin: OpenClaw install, Hermes Gateway sidecar, L0–L3 pipeline, context offload, storage backends, tools, CLI, and operations.
## Context Links
- [Agent index](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/llms.txt)
- [Human interactive docs](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97)
- [GitHub repository](https://github.com/TencentCloud/TencentDB-Agent-Memory)
## Repository Metadata
- Repository: TencentCloud/TencentDB-Agent-Memory
- Generated: 2026-07-09T07:43:00.201Z
- Updated: 2026-07-09T07:43:41.221Z
- Runtime: Grok CLI
- Format: Documentation
- Pages: 21
## Page Index
- 01. [Overview](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/01-overview.md) - What TencentDB Agent Memory exposes, who should use it, host entry points (OpenClaw plugin, Hermes provider, Gateway), and the first docs routes to follow.
- 02. [Installation](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/02-installation.md) - Prerequisites (Node >= 22.16.0, OpenClaw >= 2026.3.13), openclaw plugins install/update, link-from-source development install, and postinstall patch behavior.
- 03. [Quickstart](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/03-quickstart.md) - Zero-config enable in openclaw.json, gateway restart, success signals ([memory-tdai] logs, data dir layout), and first smoke checks with tdai tools.
- 04. [Memory layers](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/04-memory-layers.md) - Repo-specific L0 conversation, L1 atom, L2 scene, and L3 persona model: producers, storage shapes, pipeline triggers, and drill-down paths.
- 05. [Context offload](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/05-context-offload.md) - Symbolic short-term memory: tool-pair offload, L1/L1.5/L2 Mermaid canvas, L3 injection thresholds, node_id drill-down, and independent enable defaults.
- 06. [Host adapters](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/06-host-adapters.md) - Host-neutral TdaiCore boundary: RuntimeContext, HostAdapter, LLMRunnerFactory, and OpenClaw vs standalone/Hermes Gateway adapter paths.
- 07. [Configure storage backends](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/07-configure-storage-backends.md) - Choose sqlite (sqlite-vec + FTS5) or tcvdb, required tcvdb fields, embedding provider quadruplets, BM25 language, and verification of store health.
- 08. [Enable context offload](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/08-enable-context-offload.md) - Turn on offload.enabled, register plugins.slots.contextEngine, apply after-tool-call patch, and confirm Mermaid injection and data under context-offload.
- 09. [Install on Hermes](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/09-install-on-hermes.md) - Hermes MemoryProvider install paths (Docker greenfield, install_hermes_memory_tencentdb.sh, symlink/copy), Gateway sidecar supervision, and config.yaml provider keys.
- 10. [Seed historical conversations](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/10-seed-historical-conversations.md) - Run openclaw memory-tdai seed with Format A/B JSON input, config overrides, output directory layout, and L0→L1→L2→L3 verification signals.
- 11. [Migrate SQLite to Tencent VectorDB](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/11-migrate-sqlite-to-tencent-vectordb.md) - Build and run migrate-sqlite-to-tcvdb: dry-run, layer selection, openclaw.json rewrites, and post-migration storeBackend=tcvdb checks.
- 12. [Migrate from memory-tdai package](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/12-migrate-from-memory-tdai-package.md) - Rename path from @tdai/memory-tdai to @tencentdb-agent-memory/memory-tencentdb: backup openclaw.json, uninstall old plugin, preserve ~/.openclaw/memory-tdai data, reinstall and verify.
- 13. [Configuration reference](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/13-configuration-reference.md) - Full plugin config schema: timezone, storeBackend, capture, extraction, persona, pipeline, recall, embedding, tcvdb, bm25, report, llm, offload — defaults, constraints, and degrade rules.
- 14. [Agent tools](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/14-agent-tools.md) - Registered tools tdai_memory_search and tdai_conversation_search: parameters, hybrid/embedding/keyword strategies, RRF merge, and combined 3-calls-per-turn limit.
- 15. [Gateway HTTP API](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/15-gateway-http-api.md) - Standalone Hermes sidecar endpoints: GET /health, POST /recall, /capture, /search/memories, /search/conversations, /session/end, /seed — request/response fields and auth.
- 16. [CLI reference](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/16-cli-reference.md) - openclaw memory-tdai seed flags, and package bins migrate-sqlite-to-tcvdb, export-tencent-vdb, read-local-memory including build prerequisites.
- 17. [Data directories](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/17-data-directories.md) - On-disk layout for OpenClaw (~/.openclaw/memory-tdai), offload (~/.openclaw/context-offload), and Hermes MEMORY_TENCENTDB_ROOT trees: conversations, records, scene_blocks, vectors.db, persona, checkpoints.
- 18. [Gateway control](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/18-gateway-control.md) - memory-tencentdb-ctl start/stop/status/health/logs and config subcommands in standalone vs hermes modes, path env overrides, and LLM credential injection.
- 19. [Export diagnostics](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/19-export-diagnostics.md) - Run export-diagnostic.sh to package redacted config, gateway logs, and memory-tdai data; privacy notes and local-only delivery expectations.
- 20. [Troubleshooting](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/20-troubleshooting.md) - Source-backed failure modes: plugin silent, no recall, embedding degrade, aggressive cleanup, offload slot/patch missing, Gateway circuit breaker, and recovery probes.
- 21. [Contributing](https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/21-contributing.md) - Dev setup with openclaw plugins install --link, test scripts (vitest), build targets, commit conventions, and PR expectations.
## Source File Index
- `.github/workflows/pr-ci.yml`
- `bin/export-tencent-vdb.mjs`
- `bin/migrate-sqlite-to-tcvdb.mjs`
- `bin/read-local-memory.mjs`
- `CHANGELOG.md`
- `CONTRIBUTING.md`
- `docker/opensource/Dockerfile.hermes`
- `docker/opensource/README-hermes.md`
- `hermes-plugin/memory/memory_tencentdb/__init__.py`
- `hermes-plugin/memory/memory_tencentdb/client.py`
- `hermes-plugin/memory/memory_tencentdb/plugin.yaml`
- `hermes-plugin/memory/memory_tencentdb/README.md`
- `hermes-plugin/memory/memory_tencentdb/supervisor.py`
- `hermes-plugin/memory/memory_tencentdb/tests/test_memory_tencentdb_recovery.py`
- `index.ts`
- `openclaw.plugin.json`
- `package.json`
- `README.md`
- `scripts/bugfix-20260423/BUGFIX-20260423-SOP.md`
- `scripts/export-diagnostic.sh`
- `scripts/install_hermes_memory_tencentdb.sh`
- `scripts/memory-tencentdb-ctl.sh`
- `scripts/migrate-sqlite-to-tcvdb/cli-entry.ts`
- `scripts/migrate-sqlite-to-tcvdb/config-write.ts`
- `scripts/migrate-sqlite-to-tcvdb/README.md`
- `scripts/migrate-sqlite-to-tcvdb/sqlite-to-tcvdb.ts`
- `scripts/openclaw-after-tool-call-messages.patch.sh`
- `scripts/README.memory-tencentdb-ctl.md`
- `scripts/setup-offload.sh`
- `SKILL-DIAGNOSTIC-EXPORT.md`
- `SKILL-MIGRATION.md`
- `SKILL.md`
- `src/adapters/openclaw/host-adapter.ts`
- `src/adapters/openclaw/llm-runner.ts`
- `src/adapters/standalone/host-adapter.ts`
- `src/adapters/standalone/llm-runner.ts`
- `src/cli/commands/seed.ts`
- `src/cli/index.ts`
- `src/cli/README.md`
- `src/config.ts`
- `src/core/conversation/l0-recorder.ts`
- `src/core/hooks/auto-capture.ts`
- `src/core/hooks/auto-recall.ts`
- `src/core/persona/persona-generator.ts`
- `src/core/record/l1-extractor.ts`
- `src/core/scene/scene-extractor.ts`
- `src/core/seed/input.ts`
- `src/core/seed/seed-runtime.ts`
- `src/core/seed/types.ts`
- `src/core/store/embedding.ts`
- `src/core/store/factory.ts`
- `src/core/store/search-utils.ts`
- `src/core/store/sqlite.ts`
- `src/core/store/tcvdb.ts`
- `src/core/tdai-core.ts`
- `src/core/tools/conversation-search.ts`
- `src/core/tools/memory-search.ts`
- `src/core/types.ts`
- `src/gateway/config.ts`
- `src/gateway/server.ts`
- `src/gateway/types.ts`
- `src/offload/hooks/after-tool-call.ts`
- `src/offload/hooks/llm-input-l3.ts`
- `src/offload/index.ts`
- `src/offload/pipelines/l2-mermaid.ts`
- `src/offload/storage.ts`
- `src/offload/types.ts`
- `src/utils/checkpoint.ts`
- `src/utils/ensure-hook-policy.ts`
- `src/utils/manifest.ts`
- `src/utils/no-think-fetch.ts`
- `src/utils/openclaw-state-dir.ts`
- `src/utils/pipeline-factory.ts`
- `src/utils/pipeline-manager.ts`
- `vitest.config.ts`
- `vitest.e2e.config.ts`
---
## 01. Overview
> What TencentDB Agent Memory exposes, who should use it, host entry points (OpenClaw plugin, Hermes provider, Gateway), and the first docs routes to follow.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/01-overview.md
- Generated: 2026-07-09T07:31:53.567Z
### Source Files
- `package.json`
- `index.ts`
- `openclaw.plugin.json`
- `src/core/tdai-core.ts`
- `src/core/types.ts`
- `README.md`
---
title: "Overview"
description: "What TencentDB Agent Memory exposes, who should use it, host entry points (OpenClaw plugin, Hermes provider, Gateway), and the first docs routes to follow."
---
`@tencentdb-agent-memory/memory-tencentdb` (v0.3.6) is a host-neutral four-layer agent memory system. Core algorithms live in `TdaiCore` (`src/core/tdai-core.ts`); hosts attach through `HostAdapter` + `LLMRunnerFactory` and never embed OpenClaw or Hermes APIs inside the memory pipeline. The same core powers in-process OpenClaw registration (`index.ts`), a Node HTTP Gateway sidecar (`src/gateway/server.ts`), and the Hermes Python `memory_tencentdb` provider that talks to that Gateway.
## Package identity
| Field | Value |
| --- | --- |
| npm package | `@tencentdb-agent-memory/memory-tencentdb` |
| OpenClaw plugin id | `memory-tencentdb` |
| CLI / log alias | `memory-tdai` (`commandAliases`, log tag `[memory-tdai]`) |
| Hermes provider dir / key | `memory_tencentdb` (underscore; config alias `memory-tencentdb` / `tdai` also resolve) |
| Node | `>=22.16.0` |
| OpenClaw plugin API | `>=2026.3.13` (`package.json` `openclaw.compat`) |
| License | MIT |
Legacy package name `@tdai/memory-tdai` (plugin id `memory-tdai`) shares the on-disk data root `~/.openclaw/memory-tdai/` with the current package. Migrate via the package-rename flow; do not delete that directory when uninstalling the old plugin.
## What it exposes
Two independent memory tracks share one install:
| Track | Default | Role |
| --- | --- | --- |
| **Layered long-term memory** | On when capture/recall/extraction defaults apply | L0 conversations → L1 atoms → L2 scene blocks → L3 persona; auto-recall before each turn; agent tools for search |
| **Symbolic short-term memory (context offload)** | `offload.enabled: false` | Offloads tool-pair logs, builds L1/L1.5/L2 Mermaid canvas, injects compact symbols with `node_id` drill-down |
### Long-term layers (L0–L3)
| Layer | Producer | Typical storage under data dir |
| --- | --- | --- |
| **L0 Conversation** | Auto-capture on successful turn end | `conversations/*.jsonl` + vector index |
| **L1 Atom** | LLM extraction + optional vector/keyword dedup | `records/*.jsonl` + vector index |
| **L2 Scene** | Pipeline after L1 (Markdown scene blocks) | `scene_blocks/*.md` |
| **L3 Persona** | Triggered every N new L1 memories (default 50) | `persona.md` (+ backups) |
Recall injects dynamic L1 hits as user-prompt `prependContext` and stable persona / scene navigation / tool guidance as `appendSystemContext` (system prompt). Capture records L0 and notifies the L1→L2→L3 pipeline scheduler.
### Agent tools (OpenClaw)
Registered when capture or recall is enabled:
| Tool | Searches | Notes |
| --- | --- | --- |
| `tdai_memory_search` | L1 structured memories | Hybrid / embedding / keyword strategies |
| `tdai_conversation_search` | L0 raw conversations | Use when L1 is insufficient |
Both tools share a **combined limit of 3 calls per turn**.
### Gateway HTTP API (Hermes sidecar)
Node `http` server (default `127.0.0.1:8420`, no Express/Fastify):
| Method | Path | Maps to |
| --- | --- | --- |
| `GET` | `/health` | Liveness (no auth required) |
| `POST` | `/recall` | `TdaiCore.handleBeforeRecall` |
| `POST` | `/capture` | `TdaiCore.handleTurnCommitted` |
| `POST` | `/search/memories` | `TdaiCore.searchMemories` |
| `POST` | `/search/conversations` | `TdaiCore.searchConversations` |
| `POST` | `/session/end` | `TdaiCore.handleSessionEnd` (per-session flush only) |
| `POST` | `/seed` | Batch historical seed (L0→L1) |
Optional `TDAI_GATEWAY_API_KEY` enforces `Authorization: Bearer` on all routes except `GET /health`.
### CLI and package bins
| Surface | Purpose |
| --- | --- |
| `openclaw memory-tdai seed` | Seed historical conversations into the memory tree |
| `migrate-sqlite-to-tcvdb` | SQLite → Tencent VectorDB migration |
| `export-tencent-vdb` | Export Tencent VectorDB content |
| `read-local-memory` | Inspect local memory files |
| `scripts/memory-tencentdb-ctl.sh` | Gateway start/stop/status/health/logs/config |
| `scripts/install_hermes_memory_tencentdb.sh` | Hermes provider + Gateway install |
| `scripts/export-diagnostic.sh` | Redacted diagnostic package |
`postinstall` runs `scripts/openclaw-after-tool-call-messages.patch.sh` (non-fatal on failure) so offload can see `after-tool-call` messages when OpenClaw is present.
## Who should use it
| Audience | Fit |
| --- | --- |
| **OpenClaw operators** | Want automatic capture, layered recall, and optional token-saving offload inside the gateway process |
| **Hermes operators** | Want the same memory core via a supervised Gateway + `memory_tencentdb` MemoryProvider |
| **Integrators / hosts** | Need host-neutral `TdaiCore` APIs behind their own `HostAdapter` |
| **Maintainers** | Operate storage backends, seed history, migrate sqlite→tcvdb, or export diagnostics |
Not required: a single cloud vendor for LLM or embeddings. Extraction and embedding accept OpenAI-compatible endpoints; `storeBackend` is local `sqlite` (default) or remote `tcvdb`. Independent `llm` config can override the host model for L1/L2/L3.
## Host entry points
```mermaid
flowchart TB
subgraph hosts [Host surfaces]
OC[OpenClaw plugin
index.ts register]
HM[Hermes MemoryProvider
memory_tencentdb]
GW[TdaiGateway
src/gateway/server.ts]
end
subgraph adapters [Host adapters]
OCA[OpenClawHostAdapter]
SA[StandaloneHostAdapter]
end
subgraph core [Host-neutral core]
TC[TdaiCore]
PIPE[L0→L1→L2→L3 pipeline]
STORE[(sqlite-vec / tcvdb)]
end
subgraph offload [Optional short-term]
OFF[registerOffload
contextEngine slot]
end
OC --> OCA --> TC
HM -->|HTTP 127.0.0.1:8420| GW
GW --> SA --> TC
TC --> PIPE
TC --> STORE
OC --> OFF
```
### OpenClaw plugin
1. Install: `openclaw plugins install @tencentdb-agent-memory/memory-tencentdb`
2. Enable in `~/.openclaw/openclaw.json` (plugin id `memory-tencentdb`)
3. Restart gateway; look for `[memory-tdai]` logs and data under `~/.openclaw/memory-tdai/`
`index.ts` is a thin shell: parses config, builds `OpenClawHostAdapter` + `TdaiCore`, registers tools, hooks `before_prompt_build` (recall) and `agent_end` (capture), and optionally `registerOffload`. Runtime data path is `path.join(openclawStateDir, "memory-tdai")` (directory name kept for compatibility).
Zero-config enable:
```jsonc
// ~/.openclaw/openclaw.json
{
"memory-tencentdb": {
"enabled": true
}
}
```
Optional offload (requires `plugins.slots.contextEngine: "memory-tencentdb"` and the after-tool-call patch for best results):
```jsonc
{
"plugins": {
"slots": {
"contextEngine": "memory-tencentdb"
}
},
"memory-tencentdb": {
"config": {
"offload": {
"enabled": true
}
}
}
}
```
### Hermes provider
Python package under `hermes-plugin/memory/memory_tencentdb/`:
- **Hooks:** `on_memory_write`, `on_session_end`
- **Lifecycle map:** `prefetch` → `POST /recall`, `sync_turn` → `POST /capture` (background, max 4 in-flight), session end → `POST /session/end`
- **Reliability:** circuit breaker (5 failures → 60s pause), supervised Gateway start, auto-discovery of `src/gateway/server.ts`
- **Install paths:** Docker greenfield (`docker/opensource/Dockerfile.hermes`), `install_hermes_memory_tencentdb.sh`, or symlink/copy into Hermes `plugins/memory/memory_tencentdb`
Provider key in `config.yaml` must use the directory name:
```yaml
memory:
provider: memory_tencentdb
```
Default roots: `$MEMORY_TENCENTDB_ROOT` → `~/.memory-tencentdb` (install + data); Gateway port `8420`.
### Standalone Gateway
`TdaiGateway` constructs `StandaloneHostAdapter` (`hostType: "standalone"`, platform `"gateway"`) and one process-wide `TdaiCore`. Use for Hermes sidecar, manual ops (`memory-tencentdb-ctl`), or any HTTP client that implements the same endpoints. `handleSessionEnd` flushes **one** session only; process teardown uses `destroy()`.
## Storage and LLM boundaries
| Concern | Default | Notes |
| --- | --- | --- |
| `storeBackend` | `sqlite` | Local `vectors.db` + sqlite-vec + FTS5 |
| `storeBackend: tcvdb` | opt-in | Requires `tcvdb.url`, `apiKey`, `database` |
| Embedding | `provider: none` disables remote vectors | OpenAI-compatible quadruplet or ZeroEntropy path when set |
| Extraction LLM (OpenClaw) | Host embedded agent | Optional `llm.enabled` → standalone OpenAI-compatible runner |
| Extraction LLM (Gateway/Hermes) | `StandaloneLLMRunner` | `TDAI_LLM_*` env or `tdai-gateway.json` / yaml |
On store init failure, core degrades: pipeline can continue with JSONL fallback; recall/dedup quality drops. Logs tag degraded store paths under `[memory-tdai]`.
## On-disk layout (defaults)
```text
# OpenClaw long-term
~/.openclaw/memory-tdai/
conversations/ # L0
records/ # L1
scene_blocks/ # L2
.metadata/ .backup/
vectors.db # when sqlite backend
persona.md # L3
# OpenClaw short-term (when offload enabled)
~/.openclaw/context-offload/
# Hermes / standalone
~/.memory-tencentdb/
tdai-memory-openclaw-plugin/ # package checkout
memory-tdai/ # TDAI_DATA_DIR default
```
## `TdaiCore` capability surface
Host-neutral methods every adapter path uses:
| Method | Host mapping |
| --- | --- |
| `initialize()` / `destroy()` | Process start / `gateway_stop` |
| `handleBeforeRecall(userText, sessionKey)` | OpenClaw `before_prompt_build`, Hermes `prefetch` |
| `handleTurnCommitted(CompletedTurn)` | OpenClaw `agent_end`, Hermes `sync_turn` |
| `searchMemories` / `searchConversations` | Tools + Gateway search routes |
| `handleSessionEnd(sessionKey)` | Hermes/Gateway session end (not full destroy) |
Interfaces: `RuntimeContext`, `HostAdapter`, `LLMRunner` / `LLMRunnerFactory` in `src/core/types.ts`. Implementations: `OpenClawHostAdapter`, `StandaloneHostAdapter`.
## First path by goal
| Goal | Start here |
| --- | --- |
| Install OpenClaw plugin | [Installation](/installation) → [Quickstart](/quickstart) |
| Understand L0–L3 | [Memory layers](/memory-layers) |
| Enable short-term compression | [Enable context offload](/enable-offload) → [Context offload](/context-offload) |
| Run on Hermes | [Install on Hermes](/install-hermes) → [Gateway HTTP API](/gateway-http-api) |
| Choose sqlite vs tcvdb | [Configure storage backends](/configure-storage) |
| Seed history or migrate package | [Seed historical conversations](/seed-conversations), [Package rename](/package-rename) |
| Operate Gateway / debug | [Gateway control](/gateway-control), [Troubleshooting](/troubleshooting) |
## Next
Node and OpenClaw prerequisites, plugins install/update, link-from-source, postinstall patch.
Zero-config enable, gateway restart, success logs, first tdai tool smoke checks.
L0–L3 producers, storage shapes, pipeline triggers, drill-down paths.
TdaiCore boundary: RuntimeContext, HostAdapter, OpenClaw vs standalone/Gateway.
Docker, install script, provider symlink, Gateway supervision, config.yaml keys.
Full schema: capture, extraction, persona, pipeline, recall, embedding, tcvdb, offload.
---
## 02. Installation
> Prerequisites (Node >= 22.16.0, OpenClaw >= 2026.3.13), openclaw plugins install/update, link-from-source development install, and postinstall patch behavior.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/02-installation.md
- Generated: 2026-07-09T07:31:51.268Z
### Source Files
- `package.json`
- `openclaw.plugin.json`
- `SKILL.md`
- `CONTRIBUTING.md`
- `scripts/openclaw-after-tool-call-messages.patch.sh`
- `index.ts`
---
title: "Installation"
description: "Prerequisites (Node >= 22.16.0, OpenClaw >= 2026.3.13), openclaw plugins install/update, link-from-source development install, and postinstall patch behavior."
---
The OpenClaw host installs the published package `@tencentdb-agent-memory/memory-tencentdb` as plugin id `memory-tencentdb`. The package declares Node `engines.node` `>=22.16.0`, OpenClaw plugin compatibility `pluginApi` / `minGatewayVersion` `>=2026.3.13`, entry `openclaw.extensions` → `./index.ts` (built main `./dist/index.mjs`), and a `postinstall` that best-effort runs `scripts/openclaw-after-tool-call-messages.patch.sh` against the host OpenClaw install.
## Prerequisites
| Requirement | Constraint | Source of truth |
|---|---|---|
| Node.js | `>= 22.16.0` | `package.json` → `engines.node` |
| OpenClaw Gateway / plugin API | `>= 2026.3.13` | `package.json` → `openclaw.compat` (`pluginApi`, `minGatewayVersion`) and `openclaw.build` |
| OpenClaw peer (optional) | `openclaw` `>=2026.3.7` (optional peer) | `peerDependencies` / `peerDependenciesMeta` |
| Package manager | npm or pnpm (dev install) | Contributing docs |
Verify versions before installing:
```bash
node -v
openclaw --version
```
If either version is below the table, upgrade first. TypeScript source loading (link-from-source) relies on Node 22.16+ type stripping so OpenClaw can load `.ts` without a local build step.
Plugin id is `memory-tencentdb`. Runtime log prefix, data-directory leaf, and CLI namespace remain legacy names: `[memory-tdai]`, `~/.openclaw/memory-tdai/`, and `openclaw memory-tdai …`.
## Package identity
| Field | Value |
|---|---|
| npm package | `@tencentdb-agent-memory/memory-tencentdb` |
| Plugin id | `memory-tencentdb` |
| Manifest | `openclaw.plugin.json` (`id`, `name`, `commandAliases`, tools contracts) |
| CLI aliases | `memory-tdai` (`commandAliases` + `registerCli` in `index.ts`) |
| Registered tools | `tdai_memory_search`, `tdai_conversation_search` |
| Default store backend | `sqlite` (`sqlite-vec` + local files) |
| Activation | `activation.onStartup: true` |
Published bins (after install/build): `migrate-sqlite-to-tcvdb`, `export-tencent-vdb`, `read-local-memory`.
## Install on OpenClaw (release)
Use the OpenClaw plugin CLI so the host registers the package and runs lifecycle hooks:
```bash
openclaw plugins install @tencentdb-agent-memory/memory-tencentdb
```
Prefer this over a bare global `npm install` into an unrelated tree; the host plugin registry is what loads `memory-tencentdb` at gateway start.
```bash
openclaw gateway restart
```
Edit `~/.openclaw/openclaw.json` (or the path resolved by `OPENCLAW_CONFIG_PATH` / `OPENCLAW_STATE_DIR`):
```json
{
"memory-tencentdb": {
"enabled": true
}
}
```
With only `enabled: true`, defaults apply: local SQLite backend, capture/extraction/recall on, offload off. Full schema lives under `openclaw.plugin.json` → `configSchema`.
- Gateway logs contain the `[memory-tdai]` tag (plugin register / config parse).
- Data root exists at `{stateDir}/memory-tdai` — usually `~/.openclaw/memory-tdai` (`resolveOpenClawStateDir` + `"memory-tdai"` in `index.ts`).
- Expected subdirs after init: `conversations/`, `records/`, `scene_blocks/`, `.metadata/`, `.backup/` (`initDataDirectories`); vector file `vectors.db` appears once the sqlite store initializes.
### Update
Always update through OpenClaw’s plugin commands. That path avoids disabling the plugin when a semantic version range is rewritten by a generic package manager:
```bash
openclaw plugins update @tencentdb-agent-memory/memory-tencentdb
# equivalent short form used in setup skill:
openclaw plugins update memory-tencentdb
openclaw gateway restart
```
After OpenClaw itself is upgraded, re-run the after-tool-call patch (below) if you use context offload.
### Uninstall / rename path
Uninstalling removes plugin config entries from `openclaw.json` but does **not** delete `~/.openclaw/memory-tdai/`. Migrating from the old package `@tdai/memory-tdai` is a separate procedure; see [Migrate from memory-tdai package](/package-rename).
## Link-from-source (development)
No `npm run build` is required for day-to-day plugin development: Node 22.16+ strips types, and OpenClaw loads the extension path `./index.ts` from the package `openclaw.extensions` field.
```bash
git clone https://github.com/TencentCloud/TencentDB-Agent-Memory.git
cd TencentDB-Agent-Memory
npm install
openclaw plugins install --link .
openclaw gateway restart
```
| Step | Behavior |
|---|---|
| `npm install` | Installs dependencies; runs `postinstall` patch against the host OpenClaw install (best-effort). |
| `openclaw plugins install --link .` | Registers the current working tree as the live plugin. |
| Edit source | Restart gateway; linked tree is reloaded without reinstall. |
| `npm test` | `vitest run` (see contributing). |
| Packaging / bins | `npm run build` produces `dist/index.mjs` and script bins when publishing or using migrate/export CLIs. |
`package.json` `files` includes `src/`, `index.ts`, patch/setup scripts, `hermes-plugin/`, and `openclaw.plugin.json` for publish layouts.
## Postinstall patch behavior
### What runs
```json
"postinstall": "bash scripts/openclaw-after-tool-call-messages.patch.sh 2>/dev/null || true"
```
| Property | Behavior |
|---|---|
| Trigger | Every successful `npm install` of this package (including OpenClaw plugin install and local `npm install`). |
| Failure policy | stderr discarded; `|| true` so **install never fails** if OpenClaw is missing or patch strategies miss. |
| Manual re-run | `bash scripts/openclaw-after-tool-call-messages.patch.sh` or pass an explicit OpenClaw root: `bash scripts/openclaw-after-tool-call-messages.patch.sh /path/to/openclaw` |
| Debug | `DEBUG=1 bash scripts/openclaw-after-tool-call-messages.patch.sh` |
### Why it exists
Context offload needs the host `after_tool_call` hook event to include session messages. The script injects:
```js
messages: ctx.params.session?.messages
```
into OpenClaw’s compiled `dist/**/*.js` hookEvent construction after `durationMs`, so offload hooks can read the full tool history.
Long-term memory (L0–L3 capture/recall) does **not** require this patch. Offload does for correct tool-pair recovery.
### How the script finds OpenClaw
1. Optional CLI argument: explicit install directory.
2. Otherwise Node resolution: `which openclaw` → realpath / pnpm shim parse → walk up for `package.json` with `"name": "openclaw"`.
3. Fallback search under common global roots (`~/.local/share/pnpm`, `~/.local/node/lib/node_modules`, `/usr/local/lib/node_modules`, `/usr/lib/node_modules`).
Requires `$OPENCLAW_DIR/dist` to exist. Detected version is read from that package’s `package.json`.
### Patch strategies (first success wins, per file)
Candidates: all `dist/**/*.js` containing `after_tool_call` and an `after_tool_call`…`durationMs` context.
| Strategy | Target shape |
|---|---|
| 1 | `durationMs` then `};` then `hookRunnerAfter` / `runAfterToolCall` |
| 2 | Legacy `dispatch-*.js` with line-only `durationMs` |
| 3 | `durationMs` → `};` near `after_tool_call` without hookRunner anchor |
| 4 | Loose `hookEvent` / `hook_event` + `durationMs` fallback; restores `.pre-offload-patch.bak` if verify fails |
Idempotent: already-patched files are skipped. First backup per file: `*.pre-offload-patch.bak`.
Exit codes: `0` if any file patched or skipped as already patched; `1` if nothing matched.
### When to re-apply
- After upgrading OpenClaw (dist files are replaced).
- After a fresh OpenClaw install on a new machine path.
- If offload is enabled but tool pairs never land — verify injection with the patch script or `setup-offload.sh` (which hard-requires a successful patch).
Offload enablement (slot + `offload.enabled`) is documented under [Enable context offload](/enable-offload). One-shot helper: `scripts/setup-offload.sh --enable …` runs the same patch, then writes `plugins.slots.contextEngine` and offload config.
## Install surfaces outside OpenClaw
| Host | Install surface | Docs |
|---|---|---|
| Hermes Agent | Docker image, `scripts/install_hermes_memory_tencentdb.sh`, or copy/symlink `hermes-plugin/memory/memory_tencentdb` + Node Gateway sidecar | [Install on Hermes](/install-hermes) |
| Standalone Gateway | Package tree under `~/.memory-tencentdb/…` + `src/gateway/server.ts` | [Gateway control](/gateway-control), [Gateway HTTP API](/gateway-http-api) |
This page covers the OpenClaw plugin path only.
## Install graph
```text
Node >= 22.16.0 OpenClaw >= 2026.3.13
\ /
v v
openclaw plugins install @tencentdb-agent-memory/memory-tencentdb
|
+--> npm lifecycle: postinstall
| |
| v
| openclaw-after-tool-call-messages.patch.sh
| (best-effort; injects messages into after_tool_call)
|
v
openclaw.plugin.json id=memory-tencentdb
openclaw.extensions -> ./index.ts (or dist/index.mjs when built)
|
v
openclaw gateway restart
|
+--> register() in index.ts log tag [memory-tdai]
+--> pluginDataDir = {stateDir}/memory-tdai
+--> tools: tdai_memory_search, tdai_conversation_search
```
## Troubleshooting install
| Symptom | Check |
|---|---|
| Plugin silent after install | `memory-tencentdb.enabled` is true; gateway restarted; `openclaw plugins list` shows the plugin loaded. |
| `npm install` succeeded but no patch | Expected when OpenClaw is not on PATH / not resolvable; run the patch script with an explicit path. Install exit is still 0. |
| Offload broken after OpenClaw upgrade | Re-run `scripts/openclaw-after-tool-call-messages.patch.sh`; confirm `plugins.slots.contextEngine` is `"memory-tencentdb"`. |
| Old package 404 / `@tdai/memory-tdai` | Use the rename migration; data under `~/.openclaw/memory-tdai/` is preserved. |
| Node too old | Engines require `>=22.16.0` (type stripping + runtime assumptions). |
| Looking for Hermes Docker / `install_hermes_memory_tencentdb.sh` | Wrong page — use [Install on Hermes](/install-hermes). |
## Next
Zero-config enable, gateway restart, `[memory-tdai]` success signals, and first `tdai_*` smoke checks.
Turn on `offload.enabled`, register `plugins.slots.contextEngine`, and confirm Mermaid injection after the postinstall patch.
Hermes MemoryProvider paths, Gateway sidecar, and provider keys.
Move from `@tdai/memory-tdai` to `@tencentdb-agent-memory/memory-tencentdb` without losing data.
Link install, vitest, build targets, and PR conventions.
Silent plugin, missing recall, offload slot/patch gaps, and recovery probes.
---
## 03. Quickstart
> Zero-config enable in openclaw.json, gateway restart, success signals ([memory-tdai] logs, data dir layout), and first smoke checks with tdai tools.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/03-quickstart.md
- Generated: 2026-07-09T07:34:39.292Z
### Source Files
- `SKILL.md`
- `openclaw.plugin.json`
- `index.ts`
- `src/config.ts`
- `src/utils/pipeline-factory.ts`
- `README.md`
---
title: "Quickstart"
description: "Zero-config enable in openclaw.json, gateway restart, success signals ([memory-tdai] logs, data dir layout), and first smoke checks with tdai tools."
---
Zero-config OpenClaw enablement for `@tencentdb-agent-memory/memory-tencentdb` is: set plugin `enabled: true` in `~/.openclaw/openclaw.json`, restart the gateway, then confirm runtime via `[memory-tdai]` log lines and the on-disk tree under `~/.openclaw/memory-tdai/`. `parseConfig({})` supplies full defaults — capture, extraction, recall, and the L1→L2→L3 pipeline start without any embedding or offload keys.
## Prerequisites
| Requirement | Constraint |
| :--- | :--- |
| Node.js | `>= 22.16.0` |
| OpenClaw | `>= 2026.3.13` |
| Plugin package | `@tencentdb-agent-memory/memory-tencentdb` (plugin id `memory-tencentdb`) |
| Install | `openclaw plugins install @tencentdb-agent-memory/memory-tencentdb` (see [Installation](/installation)) |
```bash
openclaw --version
node -v
openclaw plugins list | grep -i memory
```
CLI command alias remains `memory-tdai` (`openclaw.plugin.json` `commandAliases`). Log prefix is always `[memory-tdai]` even though the package id is `memory-tencentdb`.
## Zero-config enable
Plugin config is empty-object safe: every field has a default. Minimal host enablement only needs the plugin entry switched on.
### 1. Write minimal `openclaw.json`
Edit `~/.openclaw/openclaw.json`. Prefer the OpenClaw `plugins.entries` shape used by migration tooling; the short form in the README is equivalent when OpenClaw maps it to the same entry.
```json
{
"plugins": {
"entries": {
"memory-tencentdb": {
"enabled": true
}
}
}
}
```
```json
{
"memory-tencentdb": {
"enabled": true
}
}
```
With only `enabled: true`, the plugin receives an empty (or default) `pluginConfig` and `parseConfig` resolves:
| Surface | Default | Effect on first run |
| :--- | :--- | :--- |
| `storeBackend` | `sqlite` | Local `vectors.db` + FTS5 under the plugin data dir |
| `capture.enabled` | `true` | L0 JSONL on successful `agent_end` |
| `extraction.enabled` | `true` | Background L1 extraction |
| `recall.enabled` | `true` | Auto-recall on `before_prompt_build` |
| `recall.strategy` | `hybrid` | FTS + embedding when available; FTS-only when embedding is off |
| `embedding.provider` | `none` | Vector search disabled until a remote quadruplet is set |
| `offload.enabled` | `false` | Short-term context offload stays off |
| `pipeline.everyNConversations` | `5` | L1 batch cadence (warmup starts at 1) |
Optional tuning (`capture`, `extraction`, `pipeline`, `recall`, `persona`, `embedding`, …) lives in the plugin config object passed to `api.pluginConfig` / `parseConfig` — not required for first light-up. Full schema: [Configuration reference](/configuration-reference).
### 2. Restart the gateway
```bash
openclaw gateway restart
```
`register()` runs on full gateway start (not only `cli-metadata` mode). Restart is required after enable or config changes.
### 3. Confirm success signals
#### Gateway logs (`[memory-tdai]`)
Watch OpenClaw gateway logs for the plugin tag. Expected patterns after a healthy start and a short conversation:
| Phase | Example / signal | Meaning |
| :--- | :--- | :--- |
| Register | `[memory-tdai] Registering plugin ...` | Entry into full register path |
| Config | `[memory-tdai] Config parsed: capture=true, recall=true(...), extraction=true(...)` | Defaults applied |
| Data dir | `[memory-tdai] Data dir: /memory-tdai (all subdirectories initialized)` | `initDataDirectories` ran |
| Complete | `[memory-tdai] Plugin registration complete (v3.1 — TdaiCore)` | Tools/hooks/CLI wired |
| Recall (turn) | `[memory-tdai] [before_prompt_build] Recall complete (...ms), ...` | Auto-recall path ran |
| Capture (turn) | `[memory-tdai] [agent_end] Auto-capture complete (...ms), l0Recorded=..., schedulerNotified=...` | L0 write + pipeline notify |
| Tools | `[memory-tdai] [tool] tdai_memory_search called/completed` | Agent tool used |
```bash
# Adjust log path to your OpenClaw install
grep '\[memory-tdai\]' ~/.openclaw/logs/* 2>/dev/null | tail -n 50
```
If there are **no** `[memory-tdai]` lines after restart, the host is not loading the plugin. Check `plugins.entries.memory-tencentdb.enabled` (or the short-form key), that the package is installed, then restart again. See [Troubleshooting](/troubleshooting).
#### Data directory layout
State root resolution order (`resolveOpenClawStateDir`):
1. Host `runtime.state.resolveStateDir()` when present
2. `OPENCLAW_STATE_DIR`
3. Fallback `~/.openclaw`
Plugin data dir:
```text
# default: ~/.openclaw/memory-tdai
```
On register, `initDataDirectories` creates:
:::files
~/.openclaw/memory-tdai/
├── conversations/ # L0 conversation JSONL
├── records/ # L1 atom records
├── scene_blocks/ # L2 scene Markdown
├── .metadata/ # checkpoints, instance_id
└── .backup/ # backup snapshots
:::
After store init (sqlite default), also expect:
| Path | When it appears | Role |
| :--- | :--- | :--- |
| `vectors.db` | SQLite store creation | FTS5 (+ vec0 when embedding dimensions > 0) |
| `persona.md` | After L3 persona generation | L3 profile (not created on empty first boot) |
| `scene_blocks/*.md` | After L2 extraction | Scene blocks |
```bash
ls -la ~/.openclaw/memory-tdai/
# conversations/ records/ scene_blocks/ .metadata/ .backup/
# vectors.db may appear after core store init
```
Shared data path name is intentionally still `memory-tdai` for compatibility with the former package. Uninstalling/reinstalling plugins does not wipe this tree.
## Default runtime behavior (what “on” means)
```text
OpenClaw gateway
└── memory-tencentdb register()
├── parseConfig → defaults
├── initDataDirectories(~/.openclaw/memory-tdai)
├── TdaiCore + OpenClawHostAdapter
├── tools: tdai_memory_search, tdai_conversation_search
├── hooks: before_prompt_build (recall), agent_end (capture)
└── offload: only if offload.enabled
```
| Capability | Zero-config | Notes |
| :--- | :---: | :--- |
| L0 auto-capture | Yes | Skips failed turns (`agent_end` without `success`) |
| L1 / L2 / L3 pipeline | Yes | Uses host LLM unless `llm.enabled` |
| Auto-recall inject | Yes | Empty early on; grows as L0/L1 accumulate |
| Hybrid vector search | No | Needs remote `embedding` quadruplet |
| Context offload | No | Separate enable + slot + patch — [Enable context offload](/enable-offload) |
## First smoke checks
Hold 2–3 successful agent turns that include memorable facts (preferences, constraints, project names). After each success, expect:
```text
[memory-tdai] [agent_end] Auto-capture complete (...ms), l0Recorded=, schedulerNotified=...
```
Verify L0 files:
```bash
ls -la ~/.openclaw/memory-tdai/conversations/
wc -l ~/.openclaw/memory-tdai/conversations/*.jsonl 2>/dev/null
```
Start a **new** turn that should need prior context. Logs should show `before_prompt_build` recall completion. Early sessions may inject little or no L1 until the pipeline extracts atoms; persona injection appears only after L3 has written `persona.md`.
In the agent session, invoke:
| Tool | Purpose | Required param |
| :--- | :--- | :--- |
| `tdai_memory_search` | Search structured L1 memories | `query` |
| `tdai_conversation_search` | Search raw L0 dialogue | `query` |
Shared limit (tool descriptions): **combined 3 calls per turn** across both tools.
Example tool parameters for L1 search:
```json
{
"query": "preferred timezone or coding style",
"limit": 5
}
```
Optional filters on `tdai_memory_search`: `type` (`persona` | `episodic` | `instruction`), `scene`.
Success signals:
- Tool returns text results (or an empty ranked set with strategy name), not a hard registration error
- Gateway debug: `[memory-tdai] [tool] tdai_memory_search completed (...ms): total=..., strategy=...`
- Or L0: `[memory-tdai][tdai_conversation_search] RESULT (strategy=...)`
With default `embedding.provider = "none"`, search degrades to FTS/keyword paths. That is expected for zero-config. Configure remote embedding only when you need dense retrieval — [Configure storage backends](/configure-storage).
Plugin CLI namespace:
```bash
openclaw memory-tdai --help
```
Seed of historical chats is a separate workflow: [Seed historical conversations](/seed-conversations).
### Smoke checklist
| Check | Pass criteria |
| :--- | :--- |
| Plugin entry | `memory-tencentdb` / package listed and enabled |
| Restart | Gateway restarted after config write |
| Logs | Any `[memory-tdai]` registration or turn logs |
| Data dir | `~/.openclaw/memory-tdai/{conversations,records,scene_blocks}` exist |
| Capture | `l0Recorded` > 0 after successful turns, or growing JSONL under `conversations/` |
| Tools | At least one `tdai_*` call returns without tool-missing failure |
## Optional next configuration (not required for smoke)
### Recommended production groups
```json
{
"plugins": {
"entries": {
"memory-tencentdb": {
"enabled": true,
"config": {
"capture": {
"enabled": true,
"excludeAgents": [],
"l0l1RetentionDays": 90,
"cleanTime": "03:00"
},
"extraction": {
"enabled": true,
"enableDedup": true,
"maxMemoriesPerSession": 10
},
"pipeline": {
"everyNConversations": 5,
"enableWarmup": true,
"l1IdleTimeoutSeconds": 600
},
"recall": {
"enabled": true,
"maxResults": 5,
"scoreThreshold": 0.3,
"strategy": "hybrid"
},
"embedding": {
"enabled": true,
"provider": "openai",
"baseUrl": "https://api.openai.com/v1",
"apiKey": "${EMBEDDING_API_KEY}",
"model": "text-embedding-3-small",
"dimensions": 1536
}
}
}
}
}
}
```
Embedding rules:
- `provider = "none"` (default): no vectors; keyword/FTS only
- Remote provider requires **all four**: `apiKey`, `baseUrl`, `model`, `dimensions`
- Incomplete remote config logs `[memory-tdai] [EMBEDDING CONFIG ERROR] ...` and **degrades** without stopping the plugin
Retention: `l0l1RetentionDays = 0` means no cleanup; non-zero values should be `>= 3` unless `allowAggressiveCleanup` is true.
### Short-term offload (separate switch)
Offload defaults to **off** and does not affect long-term memory. Enabling needs `offload.enabled`, `plugins.slots.contextEngine = "memory-tencentdb"`, and the after-tool-call patch — full procedure on [Enable context offload](/enable-offload).
## Quick failure probes
| Symptom | Probe |
| :--- | :--- |
| Silent plugin (no `[memory-tdai]` logs) | `enabled: true`? package installed? gateway restarted? |
| Capture never runs | Only successful turns capture; failed `agent_end` is skipped |
| No recall content | Wait for L1; check `recall.enabled` / high `scoreThreshold` |
| Tools missing | Tools register when `recall.enabled \|\| capture.enabled` (both true by default) |
| Vector empty / degrade | Incomplete embedding quadruplet → non-vector mode |
| Config change ignored | Confirm `~/.openclaw/openclaw.json` path, then `openclaw gateway restart` |
Broader matrix: [Troubleshooting](/troubleshooting).
## Definition of done
Treat first enable as complete when **all** hold:
1. Plugin install/update succeeded
2. `openclaw.json` has `memory-tencentdb` enabled
3. Gateway restarted
4. `[memory-tdai]` logs visible
5. `~/.openclaw/memory-tdai/` subdirs created
6. At least one successful capture **or** tool search against local data
## Next
Prerequisites, plugins install/update, link-from-source, postinstall patch.
L0 conversation → L1 atom → L2 scene → L3 persona producers and storage.
`tdai_memory_search` / `tdai_conversation_search` parameters, strategies, RRF.
Full on-disk trees for OpenClaw, offload, and Hermes roots.
sqlite vs tcvdb, embedding quadruplets, BM25 language, store health.
Turn on short-term Mermaid offload after memory is healthy.
Silent plugin, no recall, embedding degrade, cleanup, offload slot issues.
Full schema: defaults, constraints, degrade rules.
---
## 04. Memory layers
> Repo-specific L0 conversation, L1 atom, L2 scene, and L3 persona model: producers, storage shapes, pipeline triggers, and drill-down paths.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/04-memory-layers.md
- Generated: 2026-07-09T07:35:13.667Z
### Source Files
- `src/core/conversation/l0-recorder.ts`
- `src/core/record/l1-extractor.ts`
- `src/core/scene/scene-extractor.ts`
- `src/core/persona/persona-generator.ts`
- `src/utils/pipeline-manager.ts`
- `src/core/hooks/auto-capture.ts`
- `src/core/hooks/auto-recall.ts`
---
title: "Memory layers"
description: "Repo-specific L0 conversation, L1 atom, L2 scene, and L3 persona model: producers, storage shapes, pipeline triggers, and drill-down paths."
---
`MemoryPipelineManager` schedules the long-term pyramid **L0 → L1 → L2 → L3**. Live capture enters through `performAutoCapture` (`agent_end`): L0 is written immediately, then `notifyConversation(sessionKey, [])` advances conversation counters. Extraction is **not** done in the hook; L1/L2/L3 run when the pipeline thresholds fire. Recall enters through `performAutoRecall` (before prompt build): hybrid/keyword/embedding L1 search, full L2 scene navigation, and L3 `persona.md`. Default OpenClaw data root: `~/.openclaw/memory-tdai/`.
## Layer map
| Layer | Role | Producer | Primary on-disk shape | Index / search | Downstream consumer |
| --- | --- | --- | --- | --- | --- |
| **L0 Conversation** | Incremental raw user/assistant turns | `recordConversation` via `performAutoCapture` | `conversations/YYYY-MM-DD.jsonl` (one message per line) | Optional `IMemoryStore.upsertL0` (+ deferred embed on SQLite) | L1 runner (`queryL0GroupedBySessionId` or JSONL fallback) |
| **L1 Atom** | Structured facts: `persona` / `episodic` / `instruction` | `extractL1Memories` → `writeMemory` | `records/YYYY-MM-DD.jsonl` (append-only) | Dual-write `upsertL1` + FTS/vector | L2 scene extraction; `tdai_memory_search`; auto-recall prepend |
| **L2 Scene** | Thematic Markdown scene blocks + index | `SceneExtractor.extract` (tool-enabled LLM, sandboxed to `scene_blocks/`) | `scene_blocks/*.md` + `.metadata/scene_index.json` | Profile sync optional (`type: "l2"`) | L3 persona; auto-recall `` |
| **L3 Persona** | User profile Markdown | `PersonaGenerator.generateLocalPersona` when `PersonaTrigger` fires | `persona.md` (body + appended scene nav) | Profile sync optional (`type: "l3"`) | Auto-recall `` |
**Heterogeneous storage rule:** lower layers (L0/L1) prefer DB/JSONL for retrieval and evidence; upper layers (L2/L3) prefer human-readable Markdown for high-density structure. VectorStore (SQLite or TCVDB) is the retrieval engine when healthy; JSONL remains the local append-only backup for L0/L1.
## End-to-end flow
```mermaid
flowchart TD
AE[agent_end / performAutoCapture] --> L0W[L0 JSONL + optional L0 vectors]
L0W --> N[notifyConversation]
N -->|count >= threshold or idle timeout| L1Q[SerialQueue L1]
L1Q --> L1X[extractL1Memories + writeMemory]
L1X --> L1S[records/*.jsonl + upsertL1]
L1X -->|delayAfterL1 + min/max interval| L2Q[SerialQueue L2]
L2Q --> L2X[SceneExtractor on scene_blocks/]
L2X --> L3G{PersonaTrigger}
L3G -->|should generate| L3X[PersonaGenerator → persona.md]
BP[before_prompt_build / performAutoRecall] --> R1[L1 hybrid search]
BP --> R2[scene_index → navigation]
BP --> R3[persona.md body]
R1 --> PRE[prependContext relevant-memories]
R2 --> SYS[appendSystemContext]
R3 --> SYS
```
Wiring: `createPipeline` / `createL1Runner` / `createL2Runner` / `createL3Runner` in the pipeline factory attach runners to `MemoryPipelineManager`. The same factories serve live OpenClaw registration and `seed` CLI.
## L0 — Conversation
**Producer:** `performAutoCapture` holds a per-session file lock via `CheckpointManager.captureAtomically`, then calls `recordConversation`.
**Incremental capture (dual guard):**
1. **Position slice** — `originalUserMessageCount` from `before_prompt_build` keeps only messages appended after that prompt.
2. **Timestamp cursor** — `afterTimestamp` (checkpoint max timestamp; cold start may floor on plugin start time) keeps `timestamp > cursor`.
**Sanitization:** strip injected tags (`sanitizeText`), strip assistant fenced code blocks, drop noise via `shouldCaptureL0`. Polluted user text (framework `prependContext`) is replaced with cached `originalUserText` when timestamps match.
**JSONL line shape (`L0MessageRecord`):**
```json
{
"sessionKey": "agent:main:main",
"sessionId": "...",
"recordedAt": "2026-07-09T12:00:00.000Z",
"id": "msg_…",
"role": "user",
"content": "…",
"timestamp": 1720000000000
}
```
Daily shard: all sessions merge into the same `conversations/YYYY-MM-DD.jsonl`; `sessionKey` is a field, not the filename.
**Vector path:** when `vectorStore` is present, each message becomes an `L0Record`. SQLite (`supportsDeferredEmbedding`) writes metadata/FTS first and embeds in the background; TCVDB embeds synchronously when dimensions > 0.
**Scheduler note:** capture calls `notifyConversation(sessionKey, [])` with an **empty** buffer. L1 does not consume in-memory messages; it re-reads L0 from VectorStore (primary) or JSONL (fallback) using `last_l1_cursor`.
## L1 — Atom memories
**Producer:** L1 runner groups new L0 messages by `sessionId`, then `extractL1Memories`.
**Pipeline inside one L1 run:**
1. Quality gate `shouldExtractL1` (stricter than L0).
2. Split background vs new (`maxMessagesPerExtraction` default 10, background default 5).
3. Single LLM call: scene segmentation + memory extraction (JSON mode).
4. Optional batch conflict detection against existing L1 (`enableDedup`, vector top-K = `embedding.conflictRecallTopK`, default 5).
5. Persist via `writeMemory` with actions `store` | `update` | `merge` | `skip`.
**Record shape (`MemoryRecord`):**
| Field | Meaning |
| --- | --- |
| `id` | `m__` |
| `content` | Atomic memory text |
| `type` | `persona` \| `episodic` \| `instruction` |
| `priority` | 0–100 (higher more important); `-1` = strict global instruction |
| `scene_name` | Soft scene label from L1 prompt (not yet an L2 file) |
| `source_message_ids` | L0 message `id`s that grounded this atom |
| `metadata` | e.g. episodic `activity_start_time` / `activity_end_time` |
| `timestamps` | Merge history trail |
| `createdAt` / `updatedAt` | ISO times |
| `sessionKey` / `sessionId` | Provenance |
**Storage:** append to `records/YYYY-MM-DD.jsonl`. On update/merge, VectorStore deletes target IDs immediately; JSONL stays append-only and is reconciled later by memory-cleaner.
**Cursor:** L1 advances `last_l1_cursor` to max L0 `recordedAtMs` (write time), not message timestamp — TCVDB-safe.
## L2 — Scene blocks
**Producer:** L2 runner loads L1 rows for the session with `updatedAfter` cursor (`queryMemoryRecords` / JSONL filter), then `SceneExtractor.extract`.
**LLM sandbox:** workspace is `scene_blocks/` only. System files (`.metadata/scene_index.json`, checkpoint, `persona.md`) are not visible to the tool-enabled runner.
**Scene file format:**
```markdown
-----META-START-----
created: …
updated: …
summary: …
heat: 0
-----META-END-----
… scene narrative …
```
**Index:** engineering-side `syncSceneIndex` rebuilds `.metadata/scene_index.json` from `*.md` META fields (`filename`, `summary`, `heat`, `created`, `updated`).
**Capacity policy (`persona.maxScenes`, default 15):**
- ≥ max: must MERGE before CREATE
- max − 1: UPDATE only (CREATE blocked)
- ≥ max − 3: soft prefer UPDATE/MERGE
**Out-of-band L3 signal:** extractor text may contain `[PERSONA_UPDATE_REQUEST]…[/PERSONA_UPDATE_REQUEST]` or `PERSONA_UPDATE_REQUEST: …`, which sets checkpoint `request_persona_update` for the next L3 evaluation.
**Backup:** scene directory snapshots under `.backup/` before extract (`sceneBackupCount`, default 10).
## L3 — Persona
**Producer:** `createL3Runner` → `PersonaTrigger.shouldGenerate` → `PersonaGenerator.generateLocalPersona`.
**Trigger priority (`PersonaTrigger`):**
| Priority | Condition |
| --- | --- |
| P1 | `request_persona_update` set (agent/L2 explicit request) |
| P2 | Cold start: `scenes_processed > 0`, never generated, scene files exist |
| P2.5 | Recovery: generated before but `persona.md` body empty/missing |
| P3 | First scene block: `scenes_processed === 1` and new memories since persona |
| P4 | Threshold: `memories_since_last_persona >= persona.triggerEveryN` (default **50**) |
**Generation:** reads changed scene blocks since `last_persona_time`, mode `first` | `incremental`, tool-writes `persona.md` under dataDir, then strips LLM-added nav, escapes XML tags, and appends fresh scene navigation from the index.
**Global mutex:** L3 queue concurrency = 1 with pending-flag dedup so concurrent L2 completions collapse into one persona run.
## Pipeline triggers and timers
Config group: `pipeline` (defaults from `resolveMemoryTdaiConfig`).
| Knob | Default | Effect |
| --- | --- | --- |
| `everyNConversations` | `5` | L1 when per-session `conversation_count` reaches threshold |
| `enableWarmup` | `true` | Threshold sequence `1 → 2 → 4 → … → everyN`, then steady-state |
| `l1IdleTimeoutSeconds` | `600` | Resettable idle debounce; flush below-threshold buffers |
| `l2DelayAfterL1Seconds` | `10` | After L1 success, pull L2 fire time earlier to `max(now+delay, lastL2+min)` |
| `l2MinIntervalSeconds` | `900` | Floor between L2 runs per session |
| `l2MaxIntervalSeconds` | `3600` | After L2 completes, schedule next at `now+maxInterval` if session still active |
| `sessionActiveWindowHours` | `24` | Cold sessions cancel L2 poll until next L1 |
**L1 paths:** conversation threshold · idle timeout · shutdown flush.
**L2 paths:** delay-after-L1 (downward-only timer) · maxInterval poll · shutdown flush.
**L3 path:** after L2 completion, subject to `PersonaTrigger`.
Both L1 and L2 use `SerialQueue` (concurrency 1) per manager. Pipeline session state persists through the checkpoint persister (`mergePipelineStates` → `.metadata/recall_checkpoint.json`).
## Recall injection (read path)
`performAutoRecall` races work against `recall.timeoutMs` (default 5000 ms); on timeout it skips injection rather than blocking the turn.
| Injected region | Source | XML / structure |
| --- | --- | --- |
| `prependContext` (user prefix, dynamic) | L1 search (`recall.strategy` default `hybrid`) | `…` |
| `appendSystemContext` (system tail, stable) | L3 persona body | `…` |
| | L2 full navigation from `scene_index` | `…` (absolute `Path:` when `dataDir` known) |
| | Static tools guide | `` — `tdai_memory_search`, `tdai_conversation_search`, `read_file` on scene paths; **combined max 3 tool calls per turn** |
Search strategies: `keyword` (FTS/BM25), `embedding` (cosine), `hybrid` (RRF merge). Defaults: `maxResults=5`, `scoreThreshold=0.3`.
## Drill-down paths
Progressive disclosure is intentional; high layers never replace lower evidence.
| From | To | Mechanism |
| --- | --- | --- |
| Injected L1 line / agent need | More L1 atoms | Tool `tdai_memory_search` |
| L1 atom or timeline need | Raw turns | Tool `tdai_conversation_search` (L0 vectors/FTS) |
| Scene summary in nav | Full scene narrative | `read_file` on absolute path under `scene_blocks/` |
| L1 `source_message_ids` | Grounding messages | Message `id`s recorded at L0 (`msg_…`) and referenced by L1 JSONL |
| L3 persona claim | Scenes that changed | Persona generation diffs scenes by `updated` vs `last_persona_time`; nav lists heat-sorted scenes |
| L2 heat / summary | Priority for agent attention | Navigation heat emoji bands and summary lines |
Offload Mermaid `node_id` drill-down is a **separate** short-term stack (`context-offload`), not this L0–L3 long-term pyramid.
## On-disk layout (long-term tree)
```text
~/.openclaw/memory-tdai/ # OpenClaw default (Hermes: MEMORY_TENCENTDB_ROOT)
├── conversations/YYYY-MM-DD.jsonl # L0
├── records/YYYY-MM-DD.jsonl # L1
├── scene_blocks/*.md # L2
├── persona.md # L3 + scene navigation appendix
├── vectors.db # SQLite store when storeBackend=sqlite
├── .metadata/
│ ├── recall_checkpoint.json # cursors, pipeline session state, persona triggers
│ └── scene_index.json
└── .backup/ # persona + scene snapshots
```
`initDataDirectories` ensures `conversations`, `records`, `scene_blocks`, `.metadata`, `.backup`.
## Verification signals
| Signal | Healthy meaning |
| --- | --- |
| Logs `[memory-tdai] [capture]` with `L0 recorded: N messages` | Capture wrote JSONL |
| Logs `Scheduler notified` / `Conversation threshold reached` | Pipeline counting |
| Logs `[pipeline-factory] [l1] L1 complete: extracted=… stored=…` | Atoms persisted |
| New lines under `records/` and growing VectorStore L1 rows | L1 dual-write |
| Logs `[L2] Extraction complete` and new/updated `scene_blocks/*.md` | Scenes materializing |
| Logs `[L3] Persona generation succeeded` and non-empty `persona.md` body | Profile available for recall |
| Recall log `search=…hits=… persona=…chars scene=loaded` | Injection path armed |
| Tools guide present when any memory context injects | Agent can drill down |
## Config groups that own each layer
| Config path | Layer impact |
| --- | --- |
| `capture.enabled` / `excludeAgents` / `l0l1RetentionDays` | L0 write gate and retention |
| `extraction.enabled` / `enableDedup` / `maxMemoriesPerSession` / `model` | L1 LLM extract |
| `pipeline.*` | L1/L2 scheduling |
| `persona.triggerEveryN` / `maxScenes` / `backupCount` / `sceneBackupCount` / `model` | L2 capacity + L3 cadence |
| `recall.*` | Injection strategy and budgets |
| `embedding.*` / `storeBackend` / `tcvdb.*` / `bm25.*` | Vector/FTS dual-write and search quality |
## Failure modes (layer-specific)
- **L0 empty every turn:** position slice + cursor may filter all messages; cold sessions without checkpoint can floor on plugin start time and skip pre-plugin history intentionally.
- **L1 never runs:** missing LLM runner/config, or pipeline not notified (`capture.enabled` false / no scheduler).
- **L1 store-only, no vectors:** store degraded or embedding provider `none` — JSONL still written; hybrid recall falls back.
- **L2 skipped with `skipped: true`:** no L1 rows after cursor (expected when idle).
- **L2 stuck at scene cap:** merge required when `scene_count >= maxScenes`.
- **L3 never updates:** `memories_since_last_persona` below `triggerEveryN` and no cold-start/request/recovery condition.
- **Recall timeout:** `timeoutMs` exceeded → no injection that turn; tools still available next turns.
## Related pages
Symbolic short-term stack (tool-pair offload, Mermaid canvas) independent of L0–L3.
Full OpenClaw / Hermes / offload tree layout and env overrides.
Full schema defaults for capture, extraction, persona, pipeline, recall, embedding.
`tdai_memory_search` and `tdai_conversation_search` parameters, hybrid RRF, 3-calls-per-turn limit.
Offline L0→L1→L2→L3 population via `openclaw memory-tdai seed`.
SQLite vs TCVDB binding for L0/L1 vectors and L2/L3 profile sync.
---
## 05. Context offload
> Symbolic short-term memory: tool-pair offload, L1/L1.5/L2 Mermaid canvas, L3 injection thresholds, node_id drill-down, and independent enable defaults.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/05-context-offload.md
- Generated: 2026-07-09T07:32:11.286Z
### Source Files
- `src/offload/index.ts`
- `src/offload/types.ts`
- `src/offload/hooks/llm-input-l3.ts`
- `src/offload/pipelines/l2-mermaid.ts`
- `src/offload/storage.ts`
- `src/offload/hooks/after-tool-call.ts`
- `src/config.ts`
---
title: "Context offload"
description: "Symbolic short-term memory: tool-pair offload, L1/L1.5/L2 Mermaid canvas, L3 injection thresholds, node_id drill-down, and independent enable defaults."
---
Context offload is the plugin’s **symbolic short-term memory** path: when `offload.enabled` is true, `registerOffload(api, cfg.offload)` registers OpenClaw hooks plus an `OffloadContextEngine` (`plugins.slots.contextEngine = "memory-tencentdb"`) that buffers tool pairs, summarizes them into JSONL, judges task boundaries, builds Mermaid canvases (`.mmd`), and compresses live context at fixed token-ratio thresholds. It is **orthogonal** to long-term L0–L3 memory (capture / extraction / persona / recall): offload defaults off and writes under `~/.openclaw/context-offload`, not `~/.openclaw/memory-tdai`.
Offload layer names (L1 tool summary, L1.5 task judge, L2 Mermaid, L3 compression) are **not** the same as long-term memory L0 conversation / L1 atom / L2 scene / L3 persona. Same numeric labels, different pipelines and storage.
## Independent enable defaults
| Surface | Default | Effect when off |
| --- | --- | --- |
| `offload.enabled` | `false` | `registerOffload` is not called; no offload hooks / context engine from this path |
| `capture.enabled` / `extraction.enabled` / `recall.enabled` | `true` | Unrelated; long-term memory keeps running |
| `plugins.slots.contextEngine` | unset | Even if `offload.enabled=true`, slot must be `"memory-tencentdb"` or all offload hooks become no-ops |
```jsonc
{
"plugins": {
"slots": {
"contextEngine": "memory-tencentdb"
},
"entries": {
"memory-tencentdb": {
"enabled": true,
"config": {
"offload": {
"enabled": true
}
}
}
}
}
}
```
After enable:
1. Restart the OpenClaw gateway so `registerOffload` runs.
2. Apply the runtime patch once per OpenClaw install: `bash scripts/openclaw-after-tool-call-messages.patch.sh` (re-apply after OpenClaw upgrades). Without it, `after_tool_call` often lacks `event.messages` and in-loop L3 is skipped (`patch_not_effective`).
3. Confirm logs show `[context-offload]` registration and writes under `~/.openclaw/context-offload/`.
Full enable procedure: [Enable context offload](/enable-offload).
## Architecture
```mermaid
flowchart TB
subgraph Host["OpenClaw host"]
ATC["after_tool_call / before_tool_call"]
BPB["before_prompt_build"]
CE["OffloadContextEngine.assemble"]
LI["llm_input token cache"]
end
subgraph Offload["src/offload"]
BUF["pending ToolPair buffer"]
L1["L1 summarize → OffloadEntry + refs/*.md"]
L15["L1.5 judge → long/short boundary + active MMD"]
L2["L2 generate → mmds/*.mmd + node_id backfill"]
L3["L3 mild / aggressive / emergency"]
end
subgraph Disk["~/.openclaw/context-offload/{agent}/"]
JSONL["offload-{sessionId}.jsonl"]
REFS["refs/"]
MMDS["mmds/"]
ST["state.json"]
end
ATC --> BUF
BUF --> L1
L1 --> JSONL
L1 --> REFS
BPB --> L15
CE --> L15
L15 --> MMDS
L15 --> ST
L1 --> L2
L2 --> MMDS
L2 --> JSONL
BPB --> L3
ATC --> L3
CE --> L3
JSONL --> L3
MMDS --> L3
LI --> ST
```
| Stage | Trigger | Output |
| --- | --- | --- |
| Tool pair capture | `before_tool_call` caches params; `after_tool_call` buffers pairs | In-memory `ToolPair` queue |
| L1 | Pending count ≥ `forceTriggerThreshold` (default 4), or pre-flush before L1.5 / `before_prompt_build` | `OffloadEntry` rows + `refs/*.md` full results |
| L1.5 | User turn via Context Engine `assemble` or `before_prompt_build` | `long`/`short` boundary, active MMD file, `l15Settled` |
| L2 | Null `node_id` count ≥ `l2NullThreshold` (4) **or** timeout (`l2TimeoutSeconds` 300) after L1.5 settled | Patch/write `.mmd`, backfill `node_id` (`NNN-N#`) |
| L3 | Tokens ≥ ratio × context window (mild 0.5 / aggressive 0.85 / emergency 0.95) | Replace tool bodies with summaries; delete oldest prefix; inject Mermaid |
## Data layout
Default root: `DEFAULT_DATA_ROOT` = `~/.openclaw/context-offload` (override with `offload.dataDir`).
```text
~/.openclaw/context-offload/
{agentName}/
state.json
sessions-registry.json
offload-{sessionId}.jsonl
refs/{iso-timestamp}.md
mmds/{NNN}-{label}.mmd
skills/{skillName}/SKILL.md # L4 /create-skill
```
| Path field | Isolation |
| --- | --- |
| `dataDir` | Per agent (`sessionKey` form `agent::`) |
| `offload-*.jsonl` | Per session; L2 reads **all** `offload-*.jsonl` under the agent |
| `mmds/`, `refs/`, `state.json` | Shared across sessions of the same agent |
| Worker sessions | `swebench-w{N}` suffix merges into agent name so workers get separate dirs |
### OffloadEntry schema
| Field | Type | Role |
| --- | --- | --- |
| `tool_call_id` | string | Provider tool call id (required for JSONL validation) |
| `timestamp` | string | ISO time from the tool result |
| `tool_call` | string | Compact call description |
| `summary` | string | L1 LLM summary (or `[L1 degraded]…` fallback) |
| `result_ref` | string | Relative path under agent dir, e.g. `refs/….md` |
| `node_id` | `string \| null` | `null` until L2; `"wait"` while L2 runs; then `NNN-N#` |
| `score` | number 0–10 optional | Replaceability; higher → milder L3 prefers replacement first |
| `offloaded` | boolean \| `"deleted"` optional | Mild confirmed / aggressive-emergency deleted marker |
## Tool-pair offload (capture → L1)
1. **`before_tool_call`** — cache `event.params` by `toolCallId`.
2. **`after_tool_call`** — build `ToolPair` (`toolName`, `toolCallId`, `params`, `result`, `timestamp`, …); skip heartbeats (`HEARTBEAT.md`), approval-pending results, and already-processed ids.
3. **Force L1** when pending ≥ `forceTriggerThreshold` (default **4**).
4. **L1 flush** (`flushL1`):
- Write full result to `refs/` (L1.1).
- Batch ≤ 5 pairs per LLM call (backend limit); up to 3 retries then **degraded** local entries (`node_id: null`, truncated text summary).
- Append sanitized JSONL via `appendOffloadEntries` (dedup by `tool_call_id`).
5. **Notify L2** when new null-`node_id` rows exist.
Internal memory-pipeline sessions matching `memory-.*-session-\d+` are ignored end-to-end.
## L1.5 task boundary and Mermaid canvas
L1.5 is **not** driven from L1 completion. It runs on the user-turn path (`assemble` / `before_prompt_build`):
1. Flush pending L1 pairs that belong to the **previous** turn.
2. Snapshot `startIndex = entryCounter` (boundary for this turn’s offload rows).
3. Call `l15Judge` with recent history, current MMD, and up to 10 recent MMD metas.
4. Normalize judgment: `taskCompleted`, `isContinuation`, `continuationMmdFile`, `newTaskLabel`, `isLongTask`.
5. `handleTaskTransition` creates/reactivates/clears active MMD (`NNN-{label}.mmd` with seed `flowchart TD` node).
6. Push boundary `{ startIndex, result: "long"|"short", targetMmd }`.
7. On success: `l15Settled = true`, MMD injection ready. On failure after one 3s retry: **fail-safe** short boundary, `activeMmd=null` (L2 will not attribute entries to a canvas).
8. Task switch can fire a forced L2 **flush** of residual null/`wait` rows for the **old** MMD so they are not orphaned.
### Active MMD injection
When L1.5 is settled and an active MMD exists, injectors insert a marked user message (`_mmdContextMessage: "active"`) with:
- Task goal metadata from `%%{ … }%%` header when present
- Fenced ```mermaid``` body
- Guidance that `doing` nodes are recent focus and `done` nodes are completed
Active injection runs from `injectMmdIntoMessages` (assemble / `before_prompt_build` / `llm_input` path) and incremental update on `after_tool_call` when L2 patches the file mid-loop. History MMDs are **not** injected here; they appear only after aggressive L3 deletes messages (`buildHistoryMmdInjection`).
## L2 independent Mermaid generation
L2 no longer chains directly off L1. Scheduler + `checkL2Trigger`:
| Condition | Default | Behavior |
| --- | --- | --- |
| A — null count | `l2NullThreshold = 4` | Eligible `node_id === null` under a **long** boundary ≥ threshold |
| B — timeout | `l2TimeoutSeconds = 300` | Age since last L2 (or oldest null if never ran) |
| Gate | `l15Settled` | Poll waits; force-settle after 60s if L1.5 never settles |
| Wait retry | `l2WaitRetrySeconds = 120` | `node_id === "wait"` re-eligible after wait |
| Time needs new rows | `l2TimeTriggerRequiresNewOffload = true` | Timeout path requires a null row newer than last L2 |
Eligible rows must have boundary coverage with `result === "long"` and a `targetMmd`. Heartbeat tool calls are skipped.
`runL2WithBackend` (local LLM client or remote backend):
1. Mark batch entries `node_id = "wait"`.
2. Call `l2Generate` (batches ≤ 30 entries).
3. Apply `fileAction` / `replaceBlocks` via `patchMmd`, or full `writeMmd`.
4. `backfillNodeIds` from `nodeMapping`; fallback to most-frequent mapped id or max `NNN-N#` in MMD text.
5. Failed batches leave `"wait"` for later retry.
Node id pattern: `\d{3}-N\d+` (e.g. `003-N12`).
## L3 compression thresholds
Context window resolution order: model `contextWindow` in `models.providers` → `offload.defaultContextWindow` (200000) → `PLUGIN_DEFAULTS.defaultContextWindow`.
| Phase | Threshold (default) | Action |
| --- | --- | --- |
| Below mild | tokens < `floor(window × mildOffloadRatio)` (0.5) | No compression |
| Mild | ≥ 0.5 | Score-cascade replace tool results/tool_use with L1 summaries (`score` high first); scan first `mildOffloadScanRatio` (0.7) of messages; mark `offloaded: true` |
| Aggressive | ≥ 0.85 | One-shot head delete until under threshold; protect last real user message; mark `offloaded: "deleted"`; inject history MMDs for deleted `node_id` prefixes (budget `mmdMaxTokenRatio` 0.2) |
| Emergency | ≥ 0.95 (or force after stalled aggressive) | Delete/truncate until ≤ `emergencyTargetRatio` 0.6; tool-pair-aware tail delete; in-place truncate oversized messages |
Additional runtime defaults (internal `PLUGIN_DEFAULTS`, not all exposed in `openclaw.plugin.json`):
| Key | Default |
| --- | --- |
| `aggressiveDeleteRatio` | 0.4 |
| `emergencyCompressRatio` | 0.95 |
| `emergencyTargetRatio` | 0.6 |
| `mildOffloadScanRatio` | 0.7 |
| `l3TokenCountMode` | `tiktoken` |
| `l3TiktokenEncoding` | `cl100k_base` |
L3 entry points:
| Path | Role |
| --- | --- |
| `after_tool_call` | In-loop compression when `event.messages` is present (needs patch) |
| `before_prompt_build` | Full path for Responses API / gateway HTTP (assemble may not run) |
| Context Engine `assemble` | CLI / pi-embedded path; sets per-turn flag to avoid double work |
| `mode: "collect"` | L1/L1.5/L2 still run; **L3 and context engine registration are disabled** |
### Mild replacement surface (drill-down hooks)
Replaced tool results look like:
```text
[Offloaded Tool Result | node: 003-N12]
Summary: …
result_ref: refs/….md (read this file for full tool call and raw result)
```
Tool_use blocks become compact `{ _offloaded: true, node_id, tool_call }`. The agent can read `result_ref` files on disk for full recovery.
## node_id drill-down path
```text
Mermaid canvas node 003-N12
│
▼
offload-*.jsonl rows with node_id = "003-N12"
(tool_call, summary, score, tool_call_id)
│
▼
result_ref → refs/{timestamp}.md
(full sanitized tool result)
```
| Layer | Location | Use |
| --- | --- | --- |
| Symbol | `mmds/{NNN}-{label}.mmd` nodes `NNN-N#` | Task orientation in context |
| Index | JSONL `node_id` + `summary` | Mild L3 replacement text; L4 skill generation filters by node set in MMD |
| Evidence | `result_ref` MD | Lossless tool I/O recovery |
After aggressive deletion, `buildHistoryMmdInjection` re-injects non-active MMDs whose prefix matches deleted entries’ `node_id` prefixes, so lost turns keep a symbolic canvas.
## Modes and LLM routing
| `offload.mode` | Selection | L1/L1.5/L2 | L3 | Context engine |
| --- | --- | --- | --- | --- |
| `local` (default if no `backendUrl`) | `LocalLlmClient` via `offload.model` or `agents.defaults.model` + `models.providers` (apiKey / auth-profile) | Yes | Yes | Registered when slot matches |
| `backend` | `BackendClient` requires `backendUrl` | Via remote service | Yes | Registered when slot matches |
| `collect` | Backend or local for async pipeline | Yes (async) | **No** | **Not** registered (legacy compaction) |
If the LLM client cannot be constructed, logs warn that L1/L1.5/L2/L4 are disabled while L3 compression may still run when summaries already exist.
BYOK/BYOC: local mode uses the host’s configured OpenAI-compatible providers only; no hard-coded vendor is required. Backend mode is optional remote offload for summarization/judge/canvas generation.
## Configuration keys
Parsed under plugin config `offload` (`parseConfig`):
| Key | Default | Notes |
| --- | --- | --- |
| `enabled` | `false` | Master switch |
| `mode` | `local` (or `backend` if `backendUrl` set) | `local` \| `backend` \| `collect` |
| `model` | unset → main agent model | `provider/model-id` |
| `temperature` | `0.2` | Local offload LLM |
| `disableThinking` | `false` | Local mode only |
| `forceTriggerThreshold` | `4` | Pending pairs → L1 |
| `dataDir` | `~/.openclaw/context-offload` | Absolute path |
| `defaultContextWindow` | `200000` | Fallback window |
| `maxPairsPerBatch` | `20` | Config field; L1 HTTP batch size is also capped at 5 in code |
| `l2NullThreshold` | `4` | Null rows → L2 |
| `l2TimeoutSeconds` | `300` | Time → L2 |
| `mildOffloadRatio` | `0.5` | L3 mild |
| `aggressiveCompressRatio` | `0.85` | L3 aggressive |
| `mmdMaxTokenRatio` | `0.2` | MMD token budget |
| `backendUrl` / `backendApiKey` | unset | Backend / collect |
| `backendTimeoutMs` | `120000` (runtime parse) | Plugin schema default may show 10000; runtime uses parseConfig |
| `offloadRetentionDays` | `0` | Reclaim only if ≥ 3; values in (0,3) forced to 0 |
| `logMaxSizeMb` | `50` | Debug log reclaim |
| `userId` | machine primary IPv4 | `X-User-Id` for backend `/store` |
Full schema: [Configuration reference](/configuration-reference).
## Hooks and context engine contract
| Hook / API | Offload role |
| --- | --- |
| `api.registerContextEngine("memory-tencentdb", …)` | Required for normal mode; singleton `OffloadContextEngine` |
| `before_tool_call` | Param cache |
| `after_tool_call` | Buffer pairs, MMD update, optional L3 |
| `llm_input` | Token/system overhead cache for L1/L2 context (not primary L1.5) |
| `before_prompt_build` | L1 flush, L1.5, L3 + MMD when assemble is not used |
| `before_agent_start` | L4 `/create-skill` command detection |
| `llm_output` | Pending-count diagnostics |
If `plugins.slots.contextEngine` ≠ `"memory-tencentdb"`, or registration returns `ok: false`, `_contextEngineRejected` is set and every `api.on` handler returns immediately.
## Verification signals
| Signal | Expected |
| --- | --- |
| Gateway logs | `[context-offload] Registering…`, `Context engine registered`, L1/L2/L3 debug lines |
| Data dir | `~/.openclaw/context-offload//offload-*.jsonl` growing after tools |
| Refs | `refs/*.md` after successful L1 |
| MMD | `mmds/*.mmd` after long-task L1.5 + L2 |
| node_id | Transitions `null` → `wait` → `NNN-N#` |
| Mild L3 | Tool results contain `[Offloaded Tool Result \| node: …]` |
| Patch health | Absence of `after_tool_call patch check: NOT EFFECTIVE` |
| Slot misconfig | `Context engine slot not assigned… ALL offload functions disabled` |
## Failure modes
| Symptom | Cause | Recovery |
| --- | --- | --- |
| No offload activity | `offload.enabled` false | Set true, restart |
| Hooks silent / no L1.5 settle | Wrong or missing `contextEngine` slot | Set to `memory-tencentdb` |
| No in-loop L3 | Missing after-tool-call patch | Run `scripts/openclaw-after-tool-call-messages.patch.sh` |
| L1/L2 idle, L3 only | No model / backend credentials | Fix `offload.model` or `backendUrl` + providers |
| L2 never runs | L1.5 short / fail-safe; only short boundaries | Long tasks only get MMD; check L1.5 logs |
| Entries stuck on `wait` | L2 batch failure | Wait ≥ `l2WaitRetrySeconds` or fix LLM backend |
| Aggressive “stalled” | Last user message blocks head delete | Emergency force path |
| Slot conflict | Another context engine owns the slot | Free slot or disable other engine |
## Related pages
Turn on offload.enabled, set contextEngine slot, apply after-tool-call patch, verify Mermaid and data.
Long-term L0–L3 (conversation, atom, scene, persona) — separate from offload L1–L3.
On-disk trees for memory-tdai vs context-offload.
Full offload.* schema, defaults, and degrade rules.
Offload slot/patch missing and related recovery probes.
---
## 06. Host adapters
> Host-neutral TdaiCore boundary: RuntimeContext, HostAdapter, LLMRunnerFactory, and OpenClaw vs standalone/Hermes Gateway adapter paths.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/06-host-adapters.md
- Generated: 2026-07-09T07:32:39.529Z
### Source Files
- `src/core/types.ts`
- `src/core/tdai-core.ts`
- `src/adapters/openclaw/host-adapter.ts`
- `src/adapters/standalone/host-adapter.ts`
- `src/adapters/openclaw/llm-runner.ts`
- `src/adapters/standalone/llm-runner.ts`
- `index.ts`
---
title: "Host adapters"
description: "Host-neutral TdaiCore boundary: RuntimeContext, HostAdapter, LLMRunnerFactory, and OpenClaw vs standalone/Hermes Gateway adapter paths."
---
`TdaiCore` is the single host-neutral facade for recall, capture, search, pipeline scheduling, and session flush. Hosts never call extraction or store internals directly; they implement `HostAdapter` + `LLMRunnerFactory` and invoke `TdaiCore` methods. Two concrete adapters ship in-tree: `OpenClawHostAdapter` (in-process OpenClaw plugin) and `StandaloneHostAdapter` (Hermes Gateway HTTP sidecar).
## Architecture boundary
```mermaid
flowchart TB
subgraph hosts [Host shells]
OC["index.ts OpenClaw plugin"]
GW["src/gateway/server.ts TdaiGateway"]
end
subgraph adapters [Adapter layer]
OHA["OpenClawHostAdapter hostType=openclaw"]
SHA["StandaloneHostAdapter hostType=standalone"]
OCR["OpenClawLLMRunnerFactory CleanContextRunner"]
SLR["StandaloneLLMRunnerFactory AI SDK OpenAI-compatible"]
end
subgraph core [Host-neutral core]
TC["TdaiCore"]
HA["HostAdapter interface"]
RF["LLMRunnerFactory / LLMRunner"]
RC["RuntimeContext"]
end
OC --> OHA
GW --> SHA
OHA --> HA
SHA --> HA
OHA --> OCR
SHA --> SLR
OCR --> RF
SLR --> RF
HA --> TC
RF --> TC
RC --> HA
```
**Design rules (from `src/core/types.ts`):**
1. `TdaiCore` depends only on abstract interfaces — never on OpenClaw or Hermes APIs.
2. Each host supplies one `HostAdapter` and one `LLMRunnerFactory`.
3. `RuntimeContext` is the identity/path source for session and data directory scoping.
:::files
src/
├── core/
│ ├── types.ts # HostAdapter, RuntimeContext, LLMRunner*
│ └── tdai-core.ts # TdaiCore facade
├── adapters/
│ ├── index.ts # barrel re-exports
│ ├── openclaw/
│ │ ├── host-adapter.ts
│ │ └── llm-runner.ts # CleanContextRunner bridge
│ └── standalone/
│ ├── host-adapter.ts
│ └── llm-runner.ts # Vercel AI SDK + sandbox tools
├── gateway/
│ └── server.ts # StandaloneHostAdapter + HTTP routes
index.ts # OpenClawHostAdapter + hooks/tools
:::
## Core contracts
### `RuntimeContext`
Unified identity, platform, and path bag:
| Field | Type | Role |
| --- | --- | --- |
| `userId` | `string` | User identifier (`default_user` in CLI/OpenClaw defaults) |
| `sessionId` | `string` | Per-conversation session id |
| `sessionKey` | `string` | Stable session key for L0/L1 grouping |
| `platform` | `"openclaw" \| "hermes" \| "cli" \| "gateway" \| string` | Host platform tag |
| `agentIdentity` | `string?` | Optional agent profile name |
| `agentContext` | `"primary" \| "subagent" \| "cron" \| "flush"?` | Execution role |
| `workspaceDir` | `string` | Tool sandbox / workspace root |
| `dataDir` | `string` | Plugin data root (L0, records, scene_blocks, vectors) |
Population sources:
| Host | How context is filled |
| --- | --- |
| OpenClaw | Base defaults from adapter; per-hook via `buildRuntimeContextForSession(sessionKey, sessionId?)` |
| Hermes / Gateway | Base defaults from adapter; per-request via `buildRuntimeContextForRequest({ userId, sessionId, sessionKey, platform })` |
### `HostAdapter`
```ts
interface HostAdapter {
readonly hostType: "openclaw" | "hermes" | "standalone";
getRuntimeContext(): RuntimeContext;
getLogger(): Logger;
getLLMRunnerFactory(): LLMRunnerFactory;
}
```
| Method | Core use |
| --- | --- |
| `getRuntimeContext()` | `dataDir` for stores/pipeline; session defaults |
| `getLogger()` | All `[memory-tdai]` / `[core]` logging |
| `getLLMRunnerFactory()` | L1/L2/L3 extraction runners when standalone LLM path is selected |
| `hostType` | Rare branching (pipeline runner selection) |
### `LLMRunner` / `LLMRunnerFactory`
| Surface | Behavior |
| --- | --- |
| `LLMRunParams` | `prompt`, optional `systemPrompt`, `taskId`, `timeoutMs` (default 120_000), `maxTokens`, `workspaceDir`, `instanceId` |
| `LLMRunner.run()` | Returns text; empty string if model produces no output; throws on timeout/network/unrecoverable failure |
| `LLMRunnerCreateOptions.modelRef` | Optional full `"provider/model"` string; overrides host default |
| `LLMRunnerCreateOptions.enableTools` | Default `false` (text-only); `true` enables file tools for L2/L3 |
Tool policy:
| `enableTools` | Used by | Runner behavior |
| --- | --- | --- |
| `false` | L1 extraction, L1 dedup | Pure text |
| `true` | L2 scene, L3 persona | File tools allowed |
### `TdaiCore` public API
Construct with `{ hostAdapter, config, sessionFilter?, instanceId? }`, then `await core.initialize()`.
| Method | Host mapping | Effect |
| --- | --- | --- |
| `handleBeforeRecall(userText, sessionKey)` | OpenClaw `before_prompt_build`; Hermes `prefetch`; `POST /recall` | L1/L3 recall → `prependContext` / `appendSystemContext` |
| `handleTurnCommitted(turn)` | OpenClaw `agent_end`; Hermes `sync_turn`; `POST /capture` | L0 capture + pipeline notify |
| `searchMemories(params)` | Tool `tdai_memory_search`; `POST /search/memories` | L1 hybrid search |
| `searchConversations(params)` | Tool `tdai_conversation_search`; `POST /search/conversations` | L0 search |
| `handleSessionEnd(sessionKey)` | Hermes `on_session_end`; `POST /session/end` | Flush **one** session only |
| `destroy()` | OpenClaw `gateway_stop`; Gateway `stop()` | Full teardown |
`handleSessionEnd` must not call `destroy()`. Session end flushes one conversation while other concurrent sessions keep running. Process exit uses `destroy()` (scheduler + stores + embedding close).
## Adapter comparison
| | OpenClaw | Standalone (Gateway / Hermes) |
| --- | --- | --- |
| Class | `OpenClawHostAdapter` | `StandaloneHostAdapter` |
| `hostType` | `"openclaw"` | `"standalone"` |
| Entry | `index.ts` `register(api)` | `TdaiGateway` constructor |
| Process model | In-process plugin | Separate Node sidecar (HTTP) |
| Logger | `api.logger` | Console logger (`[tdai-gateway]`) |
| Data dir | `{openclawStateDir}/memory-tdai` | `GatewayConfig.data.baseDir` |
| Default `userId` | `"default_user"` | `defaultUserId ?? "default_user"` |
| Default `platform` | `"openclaw"` | `"gateway"` (overridable) |
| `workspaceDir` | `process.cwd()` | Same as `dataDir` |
| LLM path | `OpenClawLLMRunnerFactory` → `CleanContextRunner` / embedded agent | `StandaloneLLMRunnerFactory` → OpenAI-compatible HTTP via Vercel AI SDK |
| Session scoping helper | `buildRuntimeContextForSession` | `buildRuntimeContextForRequest` |
| Extra accessors | `getPluginApi()`, `getOpenClawConfig()`, `getPluginDataDir()` | — |
### OpenClawHostAdapter options
```ts
interface OpenClawHostAdapterOptions {
api: OpenClawPluginApi;
pluginDataDir: string; // e.g. ~/.openclaw/.../memory-tdai
openclawConfig: unknown; // model resolution for CleanContextRunner
}
```
Factory wiring: `OpenClawLLMRunnerFactory({ config, agentRuntime: api.runtime.agent, logger: api.logger })`.
### StandaloneHostAdapter options
```ts
interface StandaloneHostAdapterOptions {
dataDir: string;
llmConfig: StandaloneLLMConfig;
logger: Logger;
defaultUserId?: string;
platform?: string; // default "gateway"
}
```
```ts
interface StandaloneLLMConfig {
baseUrl: string; // e.g. https://api.openai.com/v1
apiKey: string;
model: string; // e.g. gpt-4o
maxTokens?: number; // default 4096
timeoutMs?: number; // default 120_000
disableThinking?: false | "vllm" | "deepseek" | "dashscope" | "openai" | "anthropic" | "kimi" | "gemini";
}
```
Standalone tools when `enableTools: true`: sandboxed `read_file`, `write_to_file`, `replace_in_file` under `workspaceDir` (max 20 tool iterations). Paths that escape the workspace return an error payload.
`modelRef` handling: if `opts.modelRef` is `"provider/model"`, the factory uses the segment after `/` as the OpenAI-compatible model name.
## Host entry paths
### OpenClaw (in-process)
```text
register(api)
→ parseConfig(api.pluginConfig)
→ pluginDataDir = join(resolveOpenClawStateDir(...), "memory-tdai")
→ hostAdapter = new OpenClawHostAdapter({ api, pluginDataDir, openclawConfig: api.config })
→ core = new TdaiCore({ hostAdapter, config, sessionFilter })
→ core.initialize()
→ register tools + hooks that call core.*
```
| OpenClaw surface | TdaiCore call |
| --- | --- |
| `before_prompt_build` | `core.handleBeforeRecall(userText, sessionKey)` → inject `prependContext` / `appendSystemContext` |
| `agent_end` (success) | `core.handleTurnCommitted({ userText, messages, sessionKey, ... })` |
| `tdai_memory_search` | `core.searchMemories(...)` |
| `tdai_conversation_search` | `core.searchConversations(...)` |
| `gateway_stop` | `core.destroy()` (+ cleaner shutdown) |
Shell responsibilities that stay **outside** core: prompt/message-count caches across hooks, session filter short-circuits, embedding warmup trigger, metric reporting, CLI metadata registration mode.
### Hermes Gateway (standalone sidecar)
```text
new TdaiGateway(configOverrides?)
→ loadGatewayConfig(...)
→ adapter = new StandaloneHostAdapter({ dataDir, llmConfig, logger, platform: "gateway" })
→ core = new TdaiCore({ hostAdapter: adapter, config: memory, sessionFilter })
start()
→ initDataDirectories + core.initialize()
→ http server routes → core.*
stop()
→ core.destroy()
```
| HTTP route | TdaiCore call |
| --- | --- |
| `POST /recall` | `handleBeforeRecall(query, session_key)` |
| `POST /capture` | `handleTurnCommitted({ userText, assistantText, messages, sessionKey, sessionId })` |
| `POST /search/memories` | `searchMemories` |
| `POST /search/conversations` | `searchConversations` |
| `POST /session/end` | `handleSessionEnd(session_key)` |
| `POST /seed` | seed runtime (uses core stores/config; not a HostAdapter method) |
| `GET /health` | store readiness via `getVectorStore()` / `getEmbeddingService()` |
Gateway LLM credentials resolve from `tdai-gateway.yaml` / env (`TDAI_LLM_BASE_URL`, `TDAI_LLM_API_KEY`, `TDAI_LLM_MODEL`, …). Memory behavior still uses the same `MemoryTdaiConfig` schema as the OpenClaw plugin (`parseConfig`).
## Pipeline LLM selection
`TdaiCore.wirePipelineRunners()` chooses how L1/L2/L3 call models:
```text
useStandaloneRunner =
cfg.llm.enabled OR hostAdapter.hostType !== "openclaw"
```
| Condition | L1 runner | L2/L3 runners |
| --- | --- | --- |
| OpenClaw, `llm.enabled: false` (default) | Host path via `openclawConfig` / embedded agent (`llmRunner` undefined) | Same |
| OpenClaw, `llm.enabled: true` | New `StandaloneLLMRunnerFactory` from `cfg.llm` | `enableTools: true` standalone runner |
| Standalone / Gateway | Adapter factory, `enableTools: false` | Adapter factory, `enableTools: true` |
OpenClaw plugin config override (`llm` group, defaults):
| Key | Default |
| --- | --- |
| `enabled` | `false` |
| `baseUrl` | `https://api.openai.com/v1` |
| `apiKey` | `""` |
| `model` | `gpt-4o` |
| `maxTokens` | `4096` |
| `timeoutMs` | `120000` |
| `disableThinking` | `false` |
Providers stay BYOK/BYOC: any OpenAI-compatible base URL and key work for standalone mode; OpenClaw mode uses the host’s embedded agent and models unless `llm.enabled` forces the standalone override.
## Lifecycle map
```mermaid
sequenceDiagram
participant Host as Host shell
participant Adapter as HostAdapter
participant Core as TdaiCore
participant Store as Stores / pipeline
Host->>Adapter: construct + getLogger/dataDir/factory
Host->>Core: new TdaiCore + initialize
Core->>Store: initDataDirectories + initStores + wire runners
Host->>Core: handleBeforeRecall
Core-->>Host: RecallResult inject contexts
Host->>Core: handleTurnCommitted
Core->>Store: L0 capture + scheduler notify
Host->>Core: handleSessionEnd sessionKey
Core->>Store: flushSession only
Host->>Core: destroy
Core->>Store: drain bg tasks close stores reset
```
## Verification signals
| Path | What success looks like |
| --- | --- |
| OpenClaw | Logs `[memory-tdai]` with data dir; `before_prompt_build` / `agent_end` complete; tools return search text |
| OpenClaw + `llm.enabled` | Core debug: `Using standalone LLM override: model=..., baseUrl=...` |
| Gateway start | `Gateway listening on http://{host}:{port}`; security posture log for auth/CORS |
| Gateway health | `GET /health` → `status: "ok"` when vector store ready, else `"degraded"` |
| Session end | One session flushed; other sessions’ pipeline state unchanged |
## Failure modes
| Symptom | Likely cause | Probe |
| --- | --- | --- |
| No extraction / empty L1–L3 | Standalone LLM missing `apiKey` / wrong `baseUrl` | Gateway LLM env; plugin `llm` block |
| OpenClaw pipeline still uses host agent | `llm.enabled` left `false` | Confirm config parse logs |
| Concurrent sessions wiped on end | Caller invoked `destroy` instead of `handleSessionEnd` | Use `POST /session/end` only for per-session flush |
| Recall silent | Store init failed (core continues degraded) | Health stores flags; `[memory-tdai] [core] Store init failed` |
| Gateway unauthenticated exposure | `TDAI_GATEWAY_API_KEY` unset on non-loopback bind | Startup security WARN |
## Related pages
Host entry points: OpenClaw plugin, Hermes provider, Gateway.
MemoryProvider install, Gateway sidecar supervision, config keys.
Route contracts that map to TdaiCore methods.
`memory-tencentdb-ctl` start/stop/status and LLM credential injection.
Full schema including the `llm` override group.
`tdai_memory_search` and `tdai_conversation_search` tool surface.
Silent plugin, missing recall, Gateway circuit breaker recovery.
---
## 07. Configure storage backends
> Choose sqlite (sqlite-vec + FTS5) or tcvdb, required tcvdb fields, embedding provider quadruplets, BM25 language, and verification of store health.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/07-configure-storage-backends.md
- Generated: 2026-07-09T07:32:31.184Z
### Source Files
- `src/core/store/factory.ts`
- `src/core/store/sqlite.ts`
- `src/core/store/tcvdb.ts`
- `src/core/store/embedding.ts`
- `src/config.ts`
- `openclaw.plugin.json`
---
title: "Configure storage backends"
description: "Choose sqlite (sqlite-vec + FTS5) or tcvdb, required tcvdb fields, embedding provider quadruplets, BM25 language, and verification of store health."
---
`storeBackend` selects the runtime memory index: **`sqlite`** (default local `vectors.db` with sqlite-vec + FTS5) or **`tcvdb`** (Tencent Cloud VectorDB with server-side dense embedding). `createStoreBundle()` in the store factory builds an `IMemoryStore`, an embedding service (client-side for sqlite, `NoopEmbeddingService` for tcvdb), and an optional BM25 encoder, then `init()` marks the store healthy or degraded before recall, tools, and L1 dedup use it.
## Backend choice
| `storeBackend` | Default | On-disk / remote surface | Dense vectors | Keyword / sparse | Hybrid |
| --- | --- | --- | --- | --- | --- |
| `sqlite` | yes | `{dataDir}/vectors.db` | Client embedding + sqlite-vec `vec0` | FTS5 (`l1_fts`, `l0_fts`) when available | Client RRF (not native hybrid) |
| `tcvdb` | no | TCVDB database + collections | Server-side collection embedding | Local BM25 sparse via `@tencentdb-agent-memory/tcvdb-text` | Native `hybridSearch` when BM25 is enabled |
```mermaid
flowchart TB
subgraph cfg ["Plugin config"]
SB["storeBackend"]
EMB["embedding.*"]
TCV["tcvdb.*"]
BM["bm25.*"]
end
subgraph factory ["createStoreBundle"]
SB --> SW{"sqlite | tcvdb"}
SW -->|sqlite| VS["VectorStore
vectors.db"]
SW -->|tcvdb| TS["TcvdbMemoryStore"]
EMB --> ES["OpenAI / ZeroEntropy / none"]
BM --> BE["BM25LocalEncoder or undefined"]
TCV --> TS
ES --> VS
BE --> TS
BE --> VS
TS --> NOOP["NoopEmbeddingService"]
end
subgraph init ["pipeline-factory _doInitStores"]
VS --> INIT["store.init(providerInfo)"]
TS --> INIT
INIT --> OK{"isDegraded()?"}
OK -->|yes| DEG["vectorStore = undefined
keyword / no-vector path"]
OK -->|no| LIVE["vector + FTS / hybrid ready"]
end
```
Unknown `storeBackend` values parse as **`sqlite`**. Zero-config (`{}`) uses sqlite with `embedding.provider: "none"` (keyword/FTS path only until you configure a remote embedding quadruplet).
## SQLite backend
### Config
```json
{
"storeBackend": "sqlite",
"embedding": {
"enabled": true,
"provider": "openai",
"baseUrl": "https://api.openai.com/v1",
"apiKey": "",
"model": "text-embedding-3-small",
"dimensions": 1536
},
"bm25": {
"enabled": true,
"language": "zh"
}
}
```
OpenClaw plugin placement (plugin id `memory-tencentdb`, command alias `memory-tdai`):
```json
{
"plugins": {
"entries": {
"memory-tencentdb": {
"enabled": true,
"config": {
"storeBackend": "sqlite"
}
}
}
}
}
```
### What gets created
| Artifact | Path / name | Role |
| --- | --- | --- |
| SQLite file | `{pluginDataDir}/vectors.db` | Single DB for L0/L1 metadata, vec0, FTS5 |
| L1 metadata | `l1_records` | Structured memories |
| L1 vectors | `l1_vec` (`vec0`, cosine) | Only when `embedding.dimensions > 0` |
| L0 metadata | `l0_conversations` | Raw messages |
| L0 vectors | `l0_vec` | Same dimension rule as L1 |
| FTS5 | `l1_fts`, `l0_fts` | Best-effort; jieba-segmented index text + original UNINDEXED display text |
| Embedding meta | `embedding_meta` | Provider/model/dimensions change detection |
Constructor pragmas: `busy_timeout=5000`, WAL, `cache_size=-65536`, `mmap_size=134217728`, `wal_autocheckpoint=1000`.
### Embedding dimensions and deferred vec0
| Condition | Behavior |
| --- | --- |
| `embedding.provider = "none"` (default) | `enabled` forced false; `dimensions` defaults to `0`; **vec0 tables are not created** |
| Remote provider fully configured | Client `EmbeddingService` embeds on write/search; vec0 tables use configured dimensions |
| Provider/model/dimensions change | `init()` drops vector tables, returns `needsReindex: true` for re-embed |
Factory only constructs a client embedding service when **all** of: `embedding.enabled`, `provider !== "local"`, and non-empty `apiKey`.
### Capabilities (`getCapabilities()`)
| Flag | Value |
| --- | --- |
| `vectorSearch` | `true` only when vec0 tables are ready (`dimensions > 0` and init succeeded) |
| `ftsSearch` | `true` when FTS5 tables initialized |
| `nativeHybridSearch` | always `false` |
| `sparseVectors` | always `false` |
FTS5 creation is best-effort: if unavailable, keyword search degrades; the store does not enter full degraded mode solely for missing FTS5. **sqlite-vec load failure** or schema init failure sets `isDegraded() === true` and all write/search ops become no-ops.
## Tencent VectorDB backend
### Required fields
`createStoreBundle` **throws** (caught by store init → full store unavailable) unless:
| Field | Required | Default | Notes |
| --- | --- | --- | --- |
| `tcvdb.url` | **yes** | `""` | Instance URL, e.g. `http://10.0.1.1:8100` |
| `tcvdb.apiKey` | **yes** | `""` | API key |
| `tcvdb.database` | **yes** | `""` | Must be set explicitly; unique name per instance |
| `tcvdb.username` | no | `"root"` | Account name |
| `tcvdb.embeddingModel` | no | `"bge-large-zh"` | Server-side collection embedding model |
| `tcvdb.timeout` | no | `10000` | Request timeout (ms) |
| `tcvdb.alias` | no | `""` | Optional label in `manifest.json` / store snapshot |
| `tcvdb.caPemPath` | no | — | CA PEM path for HTTPS |
```json
{
"storeBackend": "tcvdb",
"tcvdb": {
"url": "http://10.0.1.1:8100",
"username": "root",
"apiKey": "",
"database": "my_agent_memory",
"embeddingModel": "bge-large-zh",
"timeout": 10000
},
"bm25": {
"enabled": true,
"language": "zh"
}
}
```
### Collections and embedding
On `init()`, the store:
1. `createDatabase()` (idempotent; if just created, waits ~5s before collections).
2. Creates collections with database-prefixed names:
- `{database}_l1_memories` — L1 text → vector via collection embedding on field `text`
- `{database}_l0_conversations` — L0 `message_text` → vector
- `{database}_profiles` — L2/L3 profiles (embedding disabled; placeholder 1-D vector)
3. Prefers **DISK_FLAT** vector index (`dimension: 1024`, COSINE); falls back to **HNSW** on API code `15113` / unsupported DISK_FLAT messages.
4. Always indexes `sparse_vector` (inverted, IP) for hybrid/BM25 paths.
Client-side embedding is **not** used: the factory attaches `NoopEmbeddingService` (`getDimensions() === 0`, empty float arrays). Do not expect sqlite-style `embedding.*` to drive TCVDB dense vectors.
### Capabilities
| Flag | Value |
| --- | --- |
| `vectorSearch` | `true` (when not degraded) |
| `ftsSearch` | `true` if BM25 encoder present (sparse path stands in for pure FTS) |
| `nativeHybridSearch` | `true` if BM25 encoder present |
| `sparseVectors` | `true` if BM25 encoder present |
Without BM25, dense-only search remains; hybrid/sparse paths drop out.
## Embedding provider quadruplet (sqlite)
Remote embedding for **sqlite** needs a consistent quadruplet (plus optional knobs). `parseConfig` validates and **disables embedding without throwing** when fields are missing, storing `embedding.configError`.
### Required remote fields
| Field | Role |
| --- | --- |
| `embedding.provider` | Any non-`none` / non-`local` label (e.g. `openai`, `deepseek`, `azure`); `zeroentropy` uses a native wire format |
| `embedding.baseUrl` | API base URL |
| `embedding.apiKey` | Credential |
| `embedding.model` | Model id |
| `embedding.dimensions` | Positive integer matching the model |
### Provider modes
| `provider` | Result |
| --- | --- |
| `"none"` or omitted | Embedding off; dimensions `0`; vec0 deferred |
| `"local"` | Treated as disabled for user config; `configError` set; not wired from public schema |
| `"qclaw"` | Requires **`proxyUrl` + baseUrl + apiKey + model + dimensions**; requests go through the proxy with `Remote-URL` header |
| `"zeroentropy"` | `ZeroEntropyEmbeddingService` (`/v1/models/embed`); still needs the same quadruplet |
| Any other string with full quadruplet | `OpenAIEmbeddingService` (OpenAI-compatible `/embeddings`) |
### Optional knobs
| Field | Default | Purpose |
| --- | --- | --- |
| `enabled` | `true` (schema); forced `false` when provider is `none` or config invalid | Master switch |
| `sendDimensions` | `true` | Include `dimensions` in request body; set `false` for fixed-dim models (e.g. BGE-M3) that reject Matryoshka params |
| `maxInputChars` | `5000` | Truncate oversize inputs with a warning |
| `timeoutMs` | `10000` | Per-call timeout (retries up to 3 in service) |
| `recallTimeoutMs` | — | Overrides timeout on user-facing recall path |
| `captureTimeoutMs` | — | Overrides timeout on background capture/dedup path |
| `conflictRecallTopK` | `5` | Top-K for L1 conflict/dedup recall |
### Examples
```json
{
"storeBackend": "sqlite",
"embedding": {
"provider": "openai",
"baseUrl": "https://api.openai.com/v1",
"apiKey": "",
"model": "text-embedding-3-small",
"dimensions": 1536,
"sendDimensions": true
}
}
```
```json
{
"embedding": {
"provider": "openai",
"baseUrl": "http://your-host:port/v1",
"apiKey": "",
"model": "bge-m3",
"dimensions": 1024,
"sendDimensions": false
}
}
```
```json
{
"embedding": {
"provider": "qclaw",
"proxyUrl": "http://127.0.0.1:PORT",
"baseUrl": "https://upstream.example/v1",
"apiKey": "",
"model": "your-model",
"dimensions": 1024
}
}
```
```json
{
"embedding": {
"provider": "zeroentropy",
"baseUrl": "https://api.zeroentropy.dev",
"apiKey": "",
"model": "zembed-1",
"dimensions": 1536
}
}
```
Incomplete remote embedding config disables vector search for sqlite; the plugin keeps running. FTS5/keyword paths remain if available. Missing tcvdb credentials fail store creation instead of soft-disabling embedding.
## BM25 language
`bm25` configures the **local** sparse encoder (`BM25LocalEncoder` → `BM25Encoder.default(language)` from `@tencentdb-agent-memory/tcvdb-text`).
| Field | Type | Default | Values |
| --- | --- | --- | --- |
| `bm25.enabled` | boolean | `true` | When `false`, factory omits the encoder |
| `bm25.language` | string | `"zh"` | `"zh"` or `"en"` only; any other value becomes `"zh"` |
```json
{
"bm25": {
"enabled": true,
"language": "en"
}
}
```
| Backend | BM25 role |
| --- | --- |
| **tcvdb** | Encode documents on upsert (`encodeTexts`) and queries on search (`encodeQueries`); enables sparse + hybrid + `ftsSearch`/`nativeHybridSearch` capability flags |
| **sqlite** | Encoder is still constructed when enabled, but sqlite keyword path uses **FTS5 + jieba**, not sparse vectors (`sparseVectors: false`) |
Encode failures log a warning and return empty sparse vectors (dense-only search continues).
## Store lifecycle and health
### Init path
1. `createStoreBundle(cfg, { dataDir, logger })`
2. `vectorStore.init(embeddingService?.getProviderInfo())`
3. If `isDegraded()` → store and embedding cleared; pipeline falls back (keyword dedup / no vector recall)
4. On success, write or diff `/.metadata/manifest.json` store binding (first-write only; later mismatches log at debug)
### Degraded and failure signals
| Signal | Meaning |
| --- | --- |
| Log `Store created: backend=sqlite, dbPath=..., dimensions=...` | Factory succeeded |
| Log `Store created: backend=tcvdb, database=..., model=...` | Factory succeeded |
| Log `Store initialized: backend=..., provider=...` | `init()` healthy |
| Log `Store is in degraded mode, falling back to keyword dedup` | `isDegraded()` after init |
| Log `Store init failed; vector/FTS recall and dedup...` | Thrown factory error (e.g. missing tcvdb fields) or init exception |
| Log `Failed to load sqlite-vec extension` | Full sqlite store degraded |
| Log `FTS5 tables NOT available` | Keyword FTS only missing; store may still be healthy for vectors |
| `embedding.configError` | Remote embedding quadruplet incomplete; embedding disabled |
### Gateway health (Hermes / standalone sidecar)
`GET /health` (no auth):
```json
{
"status": "ok",
"version": "",
"uptime": 123,
"stores": {
"vectorStore": true,
"embeddingService": true
}
}
```
| Field | Semantics |
| --- | --- |
| `status` | `"ok"` if core has a vector store instance; else `"degraded"` |
| `stores.vectorStore` | Whether `getVectorStore()` is set |
| `stores.embeddingService` | Whether `getEmbeddingService()` is set (for tcvdb this is the noop service when store init succeeded) |
Operators can also use `memory-tencentdb-ctl health` / gateway control paths documented under Gateway control.
### Manifest store binding
After first successful init:
```json
{
"version": 1,
"createdAt": "2026-04-01T22:00:00.000Z",
"store": {
"type": "sqlite",
"sqlite": { "path": "vectors.db" }
},
"seed": null
}
```
TCVDB shape:
```json
{
"store": {
"type": "tcvdb",
"tcvdb": {
"url": "http://10.0.1.1:8100",
"database": "my_agent_memory",
"alias": "prod"
}
}
}
```
Intentional backend switches log store-binding diffs; they do not auto-migrate data. Use the SQLite→TCVDB migration tool for data movement.
### Practical verification checklist
Set `storeBackend` to `sqlite` or `tcvdb`. For tcvdb, set `url`, `apiKey`, and a unique `database` before restarting the gateway/plugin.
Provide `provider`, `baseUrl`, `apiKey`, `model`, and `dimensions`. Set `sendDimensions: false` for fixed-dimension OSS models.
Match corpus language: `bm25.language: "zh"` or `"en"`. Disable only if you intentionally want dense-only tcvdb behavior.
Expect `[memory-tdai][factory] Store created` and `Store initialized` without degraded/init-failed warnings.
Gateway: `GET /health` → `status: "ok"` and `stores.vectorStore: true`. OpenClaw sqlite: confirm `~/.openclaw/memory-tdai/vectors.db` (or host data dir) and `.metadata/manifest.json`. Exercise `tdai_memory_search` / `tdai_conversation_search` with hybrid strategy.
## Common misconfigurations
| Symptom | Likely cause | Fix |
| --- | --- | --- |
| No vector recall on sqlite | `provider: "none"` or incomplete quadruplet | Fill embedding quadruplet; restart |
| HTTP 400 on embed (matryoshka) | `sendDimensions: true` on fixed-dim model | Set `sendDimensions: false` |
| Store init failed / no vectorStore | tcvdb missing `url`/`apiKey`/`database` | Set required `tcvdb.*` fields |
| Hybrid weak on tcvdb | `bm25.enabled: false` or encode errors | Enable BM25; set correct `language` |
| Full degraded sqlite | sqlite-vec native load failed | Node ≥ 22.16 with sqlite-vec install; check extension load errors |
| Keyword-only after switch | Reindex needed after model/dim change | Allow reindex path after `needsReindex`; verify dimensions match model |
| qclaw always disabled | Missing `proxyUrl` or quadruplet field | Add all five: proxyUrl + baseUrl + apiKey + model + dimensions |
## Related pages
Full schema for storeBackend, embedding, tcvdb, bm25, recall, and degrade rules.
migrate-sqlite-to-tcvdb dry-run, layer selection, and openclaw.json rewrite.
vectors.db, conversations, records, scene_blocks, and Hermes MEMORY_TENCENTDB_ROOT trees.
tdai_memory_search / tdai_conversation_search strategies and RRF hybrid merge.
GET /health and store-backed recall/search endpoints.
Embedding degrade, silent plugin, and recovery probes.
---
## 08. Enable context offload
> Turn on offload.enabled, register plugins.slots.contextEngine, apply after-tool-call patch, and confirm Mermaid injection and data under context-offload.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/08-enable-context-offload.md
- Generated: 2026-07-09T07:33:45.908Z
### Source Files
- `src/offload/index.ts`
- `scripts/openclaw-after-tool-call-messages.patch.sh`
- `scripts/setup-offload.sh`
- `openclaw.plugin.json`
- `src/config.ts`
- `README.md`
---
title: "Enable context offload"
description: "Turn on offload.enabled, register plugins.slots.contextEngine, apply after-tool-call patch, and confirm Mermaid injection and data under context-offload."
---
Context offload is an independent short-term compression path inside `@tencentdb-agent-memory/memory-tencentdb`. When `plugins.entries.memory-tencentdb.config.offload.enabled` is true, `registerOffload()` in `src/offload/index.ts` registers hooks (`after_tool_call`, L3 compression, Mermaid injection) and, for normal modes, claims OpenClaw’s `plugins.slots.contextEngine` with id `memory-tencentdb`. Offload data defaults to `~/.openclaw/context-offload` and is separate from long-term memory under `~/.openclaw/memory-tdai`.
Offload defaults to **off**. Long-term L0–L3 memory keeps working when offload is disabled. Enabling offload does not require Tencent VectorDB; local mode uses the host agent model / `models.providers` credentials (BYOK).
## Prerequisites
| Requirement | Detail |
| :--- | :--- |
| Node | `>= 22.16.0` |
| OpenClaw | Plugin compat `>= 2026.3.13` (package peer: `openclaw >= 2026.3.7`) |
| Plugin installed | `openclaw plugins install @tencentdb-agent-memory/memory-tencentdb` (or linked from source) |
| Config file | `~/.openclaw/openclaw.json` present and valid JSON |
| LLM for L1/L1.5/L2 | **Local mode:** `offload.model` or `agents.defaults.model` as `provider/model-id`, with matching `models.providers[provider].baseUrl` + `apiKey` (or auth profile). **Backend mode:** `offload.backendUrl` (+ optional `backendApiKey`) |
## Enable checklist
Three pieces must all be true for full offload (tool-pair capture, Mermaid canvas, L3 compression via Context Engine):
| # | Surface | Required value |
| :---: | :--- | :--- |
| 1 | `plugins.entries.memory-tencentdb.config.offload.enabled` | `true` |
| 2 | `plugins.slots.contextEngine` | `"memory-tencentdb"` |
| 3 | OpenClaw `after_tool_call` messages patch | Injects `messages: ctx.params.session?.messages` into the hook event |
If the slot is missing or points elsewhere, `registerOffload` sets `_contextEngineRejected` and **all offload hooks become no-ops**. If the patch is missing, `after_tool_call` still runs but cannot reliably read session messages for L1 buffering and Mermaid injection.
```mermaid
flowchart TB
subgraph config ["~/.openclaw/openclaw.json"]
EN["offload.enabled = true"]
SLOT["plugins.slots.contextEngine = memory-tencentdb"]
MODE["offload.mode: local | backend | collect"]
end
subgraph host ["OpenClaw runtime"]
PATCH["after_tool_call patch\nmessages: session.messages"]
REG["registerOffload()"]
CE["registerContextEngine(memory-tencentdb)"]
HOOKS["hooks: after_tool_call,\nassemble / L3 / MMD inject"]
end
subgraph disk ["DEFAULT_DATA_ROOT"]
ROOT["~/.openclaw/context-offload"]
AGENT["agentName/"]
REFS["refs/*.md"]
MMD["mmds/*.mmd"]
JSONL["offload-sessionId.jsonl"]
end
EN --> REG
SLOT --> CE
PATCH --> HOOKS
REG --> CE
REG --> HOOKS
HOOKS --> REFS
HOOKS --> MMD
HOOKS --> JSONL
AGENT --> REFS
AGENT --> MMD
AGENT --> JSONL
ROOT --> AGENT
```
## Quick enable with setup script
`scripts/setup-offload.sh` is the one-shot path for **backend** offload. It backs up `openclaw.json`, runs the patch, sets the slot, writes offload fields, and sets compaction to `safeguard`.
```bash
# From the installed package or a source checkout
bash scripts/setup-offload.sh --enable \
--user-id \
--backend-url http://host:port \
[--backend-api-key ]
bash scripts/setup-offload.sh --status
bash scripts/setup-offload.sh --disable
```
| Flag | Required | Effect |
| :--- | :---: | :--- |
| `--enable` / `--disable` / `--status` | one of | Mode selection |
| `--user-id` | for enable | Written to `offload.userId` (backend `X-User-Id`) |
| `--backend-url` | for enable | Must start with `http://` or `https://` |
| `--backend-api-key` | no | Optional `offload.backendApiKey`; omitted key removes an existing key |
Enable steps (in order):
1. **Patch** — runs `scripts/openclaw-after-tool-call-messages.patch.sh`. Exit non-zero aborts enable (exit code 2).
2. **Slot** — `plugins.slots.contextEngine = "memory-tencentdb"`.
3. **Offload config** — `enabled: true`, `backendUrl`, `userId`, `backendTimeoutMs` defaulted if missing (`120000` in the script).
4. **Compaction** — `agents.defaults.compaction.mode = "safeguard"`.
After enable, **restart the OpenClaw gateway** so the slot and hooks load.
`setup-offload.sh --enable` requires `backendUrl` and is oriented to backend mode. For **local** mode (no remote offload API), use the manual config below and omit `backendUrl` so `parseConfig` resolves `mode: "local"`.
## Manual enable
### 1. Turn on offload
Edit `~/.openclaw/openclaw.json`:
```json
{
"plugins": {
"entries": {
"memory-tencentdb": {
"enabled": true,
"config": {
"offload": {
"enabled": true
}
}
}
}
}
}
```
Minimum key for registration: `offload.enabled: true`. With no `backendUrl` and no explicit `mode`, runtime mode is **`local`**.
### 2. Register the context engine slot
```json
{
"plugins": {
"slots": {
"contextEngine": "memory-tencentdb"
}
}
}
```
The plugin id and slot id must both be `memory-tencentdb` (not the legacy `openclaw-context-offload` owner string used only inside `OffloadContextEngine.info`).
### 3. Apply the after-tool-call patch
```bash
bash scripts/openclaw-after-tool-call-messages.patch.sh
# optional: path to a specific OpenClaw install
bash scripts/openclaw-after-tool-call-messages.patch.sh /path/to/openclaw
# debug failed matches
DEBUG=1 bash scripts/openclaw-after-tool-call-messages.patch.sh
```
What the patch does:
- Locates the OpenClaw package root (CLI path, pnpm shims, common global dirs).
- Finds `dist/**/*.js` files containing `after_tool_call` + `durationMs`.
- Injects `messages: ctx.params.session?.messages` into the hook event object.
- **Idempotent** — already-patched files are skipped.
- Backups: `*.pre-offload-patch.bak` next to patched files.
- Exit **0** if at least one file was patched or already patched; **1** if nothing could be patched.
Package install also runs this script from `postinstall` (`|| true`), so a failed silent postinstall does **not** block npm install — re-run the script explicitly if offload misbehaves.
### 4. Recommended companion settings
| Setting | Suggested | Why |
| :--- | :--- | :--- |
| `agents.defaults.compaction.mode` | `"safeguard"` | Set by `setup-offload.sh`; Context Engine owns compaction (`ownsCompaction: true`) |
| `offload.model` | `provider/model-id` | Local L1/L1.5/L2 when not using main agent model |
| `offload.backendUrl` | HTTPS/HTTP service | Switches auto-mode to `backend` when `mode` omitted |
| `offload.userId` | stable id | Backend store key; else primary non-loopback IPv4 |
## Modes
`parseConfig` resolves `offload.mode`:
| `mode` | When | Context Engine | L1/L1.5/L2 | L3 compression |
| :--- | :--- | :---: | :---: | :---: |
| `local` | Default if no `backendUrl` | Registered (slot required) | Local LLM via AI SDK | Yes |
| `backend` | Explicit or auto when `backendUrl` set | Registered | Remote backend | Yes |
| `collect` | Explicit `"collect"` | **Not** registered | Async collection | **No** (legacy compaction) |
In `collect` mode, if `slots.contextEngine` is still `"memory-tencentdb"`, the plugin logs a warning: the engine is not registered; remove the slot or switch mode.
## Core config keys
Runtime defaults come from `src/config.ts` (`OffloadConfig`):
| Key | Default | Notes |
| :--- | :--- | :--- |
| `enabled` | `false` | Master switch |
| `mode` | auto `local` / `backend` | Or explicit `local` \| `backend` \| `collect` |
| `model` | host default | `provider/model-id` |
| `temperature` | `0.2` | Offload LLM |
| `disableThinking` | `false` | Local mode only |
| `forceTriggerThreshold` | `4` | Pending tool pairs → force L1 |
| `dataDir` | unset | Absolute path; else `~/.openclaw/context-offload` |
| `defaultContextWindow` | `200000` | Fallback window |
| `maxPairsPerBatch` | `20` | L1 batch cap |
| `l2NullThreshold` | `4` | `node_id=null` count → L2 |
| `l2TimeoutSeconds` | `300` | Time-based L2 trigger |
| `mildOffloadRatio` | `0.5` | Mild L3 threshold (fraction of window) |
| `aggressiveCompressRatio` | `0.85` | Aggressive L3 threshold |
| `mmdMaxTokenRatio` | `0.2` | Mermaid injection token budget |
| `backendUrl` / `backendApiKey` | unset | Backend routing + auth |
| `backendTimeoutMs` | `120000` | Runtime default (schema text may differ) |
| `offloadRetentionDays` | `0` | Reclaim off unless `>= 3` |
| `logMaxSizeMb` | `50` | Debug log cap under data root |
| `userId` | auto IP fallback | Backend identity header |
## Disable
```bash
bash scripts/setup-offload.sh --disable
```
Or manually:
1. Set `offload.enabled` to `false`.
2. Delete `plugins.slots.contextEngine` (or free the slot for another engine).
3. Restart the gateway.
The after-tool-call patch is **not** reverted by disable; leave it in place or restore from `*.pre-offload-patch.bak` only if you must.
## Verify
### Status script
```bash
bash scripts/setup-offload.sh --status
```
Expect: Context Engine Slot `memory-tencentdb`, Offload enabled, backend/user fields as configured, compaction `safeguard` when enabled via the script.
### Logs
After gateway restart, look for:
| Signal | Meaning |
| :--- | :--- |
| `[context-offload] Registering offload module...` | `registerOffload` entered |
| `Context engine registered successfully` | Slot acquired |
| `LLM mode: local (...)` or `LLM mode: backend (...)` | Client selected |
| `>>> after_tool_call START` | Tool pairs buffering |
| `after_tool_call MMD: INJECTED` / `UPDATED` | Mermaid in session messages |
| `assemble CALLED` / L3 cascade logs | Context Engine assemble path |
| `Config plugins.slots.contextEngine=... ALL offload functions disabled` | **Slot misconfigured** |
| `registerContextEngine returned { ok: false` | **Slot occupied** |
| `after_tool_call patch check: NOT EFFECTIVE` | **Patch missing/failed** |
### On-disk data
Default root: `~/.openclaw/context-offload` (override with `offload.dataDir`).
:::files
~/.openclaw/context-offload/
└── /
├── state.json
├── offload-.jsonl
├── refs/
│ └── *.md
└── mmds/
└── *.mmd
:::
| Path | Role |
| :--- | :--- |
| `refs/*.md` | Full tool results (bottom layer) |
| `offload-*.jsonl` | L1 summaries + `node_id` / `result_ref` |
| `mmds/*.mmd` | Mermaid task canvas (top layer) |
| `state.json` | Active MMD, counters, session state |
After tool-heavy turns, expect new `refs` files, growing jsonl lines, and eventually `.mmd` files when L2 runs (`l2NullThreshold` / `l2TimeoutSeconds`).
### Smoke conversation
1. Restart OpenClaw gateway.
2. Run a session that executes several tools (search, shell, etc.).
3. Confirm logs show `after_tool_call` + optional MMD inject.
4. Confirm files under `~/.openclaw/context-offload//`.
5. In the next model turn, context should prefer Mermaid symbols over full tool dumps when L3 thresholds fire.
## Failure modes
| Symptom | Likely cause | Fix |
| :--- | :--- | :--- |
| No `[context-offload]` logs | `offload.enabled` false or plugin not loaded | Enable config; confirm plugin entry enabled; restart |
| All hooks no-op | Slot not `memory-tencentdb` or CE rejected | Set `plugins.slots.contextEngine`; free conflicting owner |
| Tool pairs not buffered / patch NOT EFFECTIVE | Patch not applied or OpenClaw layout unsupported | Re-run patch; `DEBUG=1`; re-apply after OpenClaw upgrade |
| L1/L1.5/L2 disabled, L3 only | Local model/provider missing or backend URL missing | Fix `offload.model` + `models.providers`, or set `backendUrl` |
| Collect mode: no compression | Expected for `mode: "collect"` | Switch to `local`/`backend` and keep the slot |
| Empty data dir | No tools yet, or hooks disabled | Exercise tool calls; check rejection logs first |
| Patch fails on every OpenClaw upgrade | dist rebuilt without injection | Re-run `openclaw-after-tool-call-messages.patch.sh` |
## Related pages
Symbolic short-term design: tool-pair offload, L1/L1.5/L2 Mermaid canvas, L3 thresholds, node_id drill-down.
Plugin install/update, Node/OpenClaw prerequisites, postinstall patch behavior.
Full offload and plugin schema, defaults, and degrade rules.
On-disk layout for memory-tdai vs context-offload trees.
Offload slot/patch failures and recovery probes.
---
## 09. Install on Hermes
> Hermes MemoryProvider install paths (Docker greenfield, install_hermes_memory_tencentdb.sh, symlink/copy), Gateway sidecar supervision, and config.yaml provider keys.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/09-install-on-hermes.md
- Generated: 2026-07-09T07:33:20.291Z
### Source Files
- `hermes-plugin/memory/memory_tencentdb/README.md`
- `hermes-plugin/memory/memory_tencentdb/__init__.py`
- `hermes-plugin/memory/memory_tencentdb/supervisor.py`
- `hermes-plugin/memory/memory_tencentdb/plugin.yaml`
- `scripts/install_hermes_memory_tencentdb.sh`
- `docker/opensource/Dockerfile.hermes`
- `docker/opensource/README-hermes.md`
---
title: "Install on Hermes"
description: "Hermes MemoryProvider install paths (Docker greenfield, install_hermes_memory_tencentdb.sh, symlink/copy), Gateway sidecar supervision, and config.yaml provider keys."
---
Hermes loads `memory_tencentdb` as a bundled `MemoryProvider` under `plugins/memory/memory_tencentdb/`. The Python package is a thin HTTP client plus process supervisor; capture, extraction, storage, and recall run in a Node.js Gateway sidecar (default `127.0.0.1:8420`) that hosts the same Core engine as the OpenClaw plugin.
```mermaid
flowchart TB
subgraph hermes["Hermes Agent (Python)"]
MM[MemoryManager]
PROV["MemoryTencentdbProvider\nhermes-plugin/memory/memory_tencentdb/"]
SUP[GatewaySupervisor]
SDK[MemoryTencentdbSdkClient]
MM --> PROV
PROV --> SUP
PROV --> SDK
end
subgraph gw["Gateway sidecar (Node.js)"]
SRV["src/gateway/server.ts"]
CORE[memory-tencentdb Core]
L0[L0 conversations]
L1[L1 atoms]
L2[L2 scene blocks]
L3[L3 persona]
SRV --> CORE
CORE --> L0
CORE --> L1
CORE --> L2
CORE --> L3
end
SUP -->|"Popen + /health (≤30s)"| SRV
SDK -->|"HTTP POST /recall /capture /search/* /session/end"| SRV
```
| Hermes lifecycle | Gateway | Behavior |
|---|---|---|
| `prefetch(query)` | `POST /recall` | Sync; injects `` text |
| `sync_turn(user, assistant)` | `POST /capture` | Background thread; max 4 in-flight |
| `shutdown` / `on_session_end` | `POST /session/end` | Flush pending pipeline work |
| `get_tool_schemas()` | — | Registers `memory_tencentdb_memory_search`, `memory_tencentdb_conversation_search` |
## Prerequisites
| Requirement | Notes |
|---|---|
| Hermes Agent installed | Bundled discovery path: `/plugins/memory/` |
| Node.js ≥ 22 | Gateway + `tsx` / `tsx/esm` launch |
| Directory name | Must be exactly `memory_tencentdb` (underscore), matching `plugin.yaml` `name` and `memory.provider` |
| LLM credentials | OpenAI-compatible endpoint for L1/L2/L3 (Gateway-side) |
Discovery requires `__init__.py` and `plugin.yaml` in the provider directory, and `__init__.py` must contain the literal string `MemoryProvider` or `register_memory_provider`.
## Choose an install path
Single container: Hermes + npm package Gateway. No local repo checkout required.
```bash
docker build -f docker/opensource/Dockerfile.hermes -t hermes-memory .
docker run -d \
--name hermes-memory \
--restart unless-stopped \
-p 8420:8420 \
-e MODEL_API_KEY="your-api-key" \
-e MODEL_BASE_URL="https://api.lkeap.cloud.tencent.com/v1" \
-e MODEL_NAME="deepseek-v3.2" \
-e MODEL_PROVIDER="custom" \
-v hermes_data:/opt/data \
hermes-memory
```
Image defaults bake Tencent Cloud DeepSeek-V3.2 for `MODEL_BASE_URL` / `MODEL_NAME` / `MODEL_PROVIDER`. You can pass only `MODEL_API_KEY` when using that endpoint.
**What the image does**
1. `npm install @tencentdb-agent-memory/memory-tencentdb@latest` under `/opt/tdai-gateway`
2. Official Hermes installer → `/usr/local/lib/hermes-agent/`
3. Symlink provider into Hermes bundled discovery:
```text
/usr/local/lib/hermes-agent/plugins/memory/memory_tencentdb
→ /opt/tdai-gateway/node_modules/@tencentdb-agent-memory/memory-tencentdb/hermes-plugin/memory/memory_tencentdb
```
4. On `CMD`: sync `MODEL_*` → `TDAI_LLM_*`, write `/opt/data/config.yaml` with `memory.provider: memory_tencentdb`, write `/opt/data/.env`, foreground-start Gateway on `:8420`
| Variable | Default | Role |
|---|---|---|
| `MODEL_API_KEY` | *(required at runtime)* | Shared LLM key for Hermes + Gateway |
| `MODEL_BASE_URL` | `https://api.lkeap.cloud.tencent.com/v1` | OpenAI-compatible base URL |
| `MODEL_NAME` | `deepseek-v3.2` | Model id |
| `MODEL_PROVIDER` | `custom` | Hermes model provider key |
| `TDAI_GATEWAY_PORT` | `8420` | Gateway listen port |
| `TDAI_GATEWAY_HOST` | `0.0.0.0` | Gateway bind address |
| `TDAI_DATA_DIR` | `/opt/data/tdai-memory` | L0–L3 data root |
| `HERMES_HOME` | `/opt/data` | Hermes config/session home |
| `MEMORY_TENCENTDB_GATEWAY_HOST` | `127.0.0.1` | Provider → Gateway host |
| `MEMORY_TENCENTDB_GATEWAY_PORT` | `8420` | Provider → Gateway port |
```bash
curl http://localhost:8420/health
docker exec -it hermes-memory hermes
```
Host install after Hermes is already present (for example after `install_hermes_ubuntu.sh`).
```bash
# As the target user (recommended)
bash scripts/install_hermes_memory_tencentdb.sh
# Or as root for another user
INSTALL_AS_USER=ubuntu bash scripts/install_hermes_memory_tencentdb.sh
```
**Path defaults** (override with env):
| Variable | Default |
|---|---|
| `MEMORY_TENCENTDB_ROOT` | `~/.memory-tencentdb` |
| `TDAI_INSTALL_DIR` | `$MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin` |
| `TDAI_DATA_DIR` | `$MEMORY_TENCENTDB_ROOT/memory-tdai` |
| `HERMES_AGENT_DIR` | `~/.hermes/hermes-agent` |
| `HERMES_HOME` | `~/.hermes` |
**Script steps**
1. **Step 0** — Migrate legacy `~/tdai-memory-openclaw-plugin` and `~/memory-tdai` into the new root (skip if both old and new exist; warn and keep new).
2. **Step 1** — `npm install @tencentdb-agent-memory/memory-tencentdb@latest` into a temp dir; copy package tree to `$TDAI_INSTALL_DIR`.
3. **Step 2** — `npm install --omit=dev` in install dir; ensure `tsx` available.
4. **Step 2.5** — Symlink provider:
```bash
ln -sf "$TDAI_INSTALL_DIR/hermes-plugin/memory/memory_tencentdb" \
"$HERMES_AGENT_DIR/plugins/memory/memory_tencentdb"
```
5. **Step 3** — **Does not auto-enable** the provider. Prints the `config.yaml` snippet if `memory.provider` is not already `memory_tencentdb`.
6. **Step 4** — Writes Gateway launch env:
- `/etc/profile.d/memory-tencentdb-env.sh` (interactive SSH)
- `~/.hermes/.env` (systemd / `load_dotenv` path — authoritative for service-managed Hermes)
Resolved `MEMORY_TENCENTDB_GATEWAY_CMD` (absolute `node` path, no runtime PATH dependency):
```bash
sh -c 'cd $TDAI_INSTALL_DIR && exec "$NODE_BIN" --import tsx/esm src/gateway/server.ts'
```
Source of truth in this repo: `hermes-plugin/memory/memory_tencentdb/`. Hermes does **not** load that path unless it is linked or copied under Hermes discovery.
**Install A — symlink (recommended while iterating)**
```bash
ln -s "$(pwd)/hermes-plugin/memory/memory_tencentdb" \
/plugins/memory/memory_tencentdb
```
**Install B — copy (frozen vendored tree)**
```bash
cp -r hermes-plugin/memory/memory_tencentdb \
/plugins/memory/memory_tencentdb
```
| Location | Precedence | Use |
|---|---|---|
| `/plugins/memory//` | Bundled (wins on name collision) | Ship path for `memory_tencentdb` |
| `$HERMES_HOME/plugins//` (default `~/.hermes/plugins/`) | User-installed | Third-party providers; not used by this package |
Gateway source (`src/gateway/`) stays in the npm/plugin checkout. The Python provider auto-discovers it or uses `MEMORY_TENCENTDB_GATEWAY_CMD`. Do not copy Gateway into hermes-agent.
## Enable in Hermes config
Edit `$HERMES_HOME/config.yaml` (default `~/.hermes/config.yaml`; Docker uses `/opt/data/config.yaml`):
```yaml
memory:
provider: memory_tencentdb # canonical
# Aliases: memory-tencentdb, tdai
```
`plugin.yaml` metadata:
| Field | Value |
|---|---|
| `name` | `memory_tencentdb` |
| `display_name` | `memory-tencentdb` |
| `aliases` | `tdai`, `memory-tencentdb` |
| `hooks` | `on_memory_write` (reserved), `on_session_end` → `POST /session/end` |
Hyphenated `memory-tencentdb` is a **config alias only**, not a valid on-disk directory name.
## Gateway sidecar supervision
`GatewaySupervisor` (`supervisor.py`) owns process lifecycle. `MemoryTencentdbProvider.initialize()`:
1. Resolves host/port/api key and `MEMORY_TENCENTDB_GATEWAY_CMD` or auto-discovery.
2. Sets `_initialized = True` immediately so tools register even before health is green.
3. If `/health` already returns `ok` or `degraded`, attaches synchronously.
4. Otherwise starts the Gateway on a daemon thread (`tdai-gateway-init`) so Hermes `__init__` is not blocked up to 30s.
5. Starts a watchdog (~10s interval) that can re-spawn a dead supervised child.
### Start modes
| Mode | When | Mechanism |
|---|---|---|
| **Auto-discovery** | `MEMORY_TENCENTDB_GATEWAY_CMD` unset | Search for `src/gateway/server.ts`; launch via `sh -c 'cd && exec pnpm exec tsx src/gateway/server.ts'` |
| **Explicit CMD** | Env set (install script writes this) | `Popen(shlex.split(cmd))`; env inherits Hermes process env |
| **External** | Gateway already listening | Health check only; no subprocess |
**Discovery order** (first existing file wins):
1. In-tree: `/src/gateway/server.ts` (from provider path `parents[3]`)
2. `~/.memory-tencentdb/tdai-memory-openclaw-plugin/src/gateway/server.ts`
3. `~/tdai-memory-openclaw-plugin/src/gateway/server.ts` (legacy)
4. `~/.hermes/plugins/tdai-memory-openclaw-plugin/src/gateway/server.ts`
### Health and crash behavior
| Parameter | Value |
|---|---|
| Health poll interval | 0.5s |
| Startup max wait | 30s |
| Acceptable `/health` status | `ok`, `degraded` |
| Child logs | `gateway.stdout.log` / `gateway.stderr.log` under log dir |
| Default log dir | `MEMORY_TENCENTDB_LOG_DIR` → else `~/.hermes/logs/memory_tencentdb` |
| Crash diagnostics | Last ~2KB of stderr log on failed startup |
| Shutdown | SIGTERM, wait 10s, then SIGKILL |
| Process group | `start_new_session=True` (detached from Hermes PG) |
### Provider reliability knobs
| Mechanism | Behavior |
|---|---|
| Circuit breaker | 5 consecutive Gateway failures → pause calls 60s |
| Capture back-pressure | Max 4 in-flight `sync_turn` threads; 5th waits ≤5s on oldest |
| Recover cooldown | ≥15s between in-flight `ensure_running` retries |
### Optional ops control
After install, day-2 start/stop/config can use `scripts/memory-tencentdb-ctl.sh` with `--hermes` / `MEMORY_TENCENTDB_MODE=hermes` so LLM env lands under `$HERMES_HOME/env.d/` for supervised children. Full subcommands: [Gateway control](/gateway-control).
## Environment and provider keys
Provider config schema (`get_config_schema`) maps to env vars:
Launch command. Env: `MEMORY_TENCENTDB_GATEWAY_CMD`. Optional if auto-discovery or external Gateway works.
Default `127.0.0.1`. Env: `MEMORY_TENCENTDB_GATEWAY_HOST`.
Default `8420` (validated 1..65535). Env: `MEMORY_TENCENTDB_GATEWAY_PORT`.
Optional client Bearer. Env: `MEMORY_TENCENTDB_GATEWAY_API_KEY` or `TDAI_GATEWAY_API_KEY`. Supervisor does **not** inject this into the child; set the same secret on the Gateway (`TDAI_GATEWAY_API_KEY` / `server.apiKey`) for enforcement.
Gateway LLM key. Env: `MEMORY_TENCENTDB_LLM_API_KEY`. Required for L1/L2/L3.
Default `https://api.openai.com/v1`. Env: `MEMORY_TENCENTDB_LLM_BASE_URL`.
Default `gpt-4o`. Env: `MEMORY_TENCENTDB_LLM_MODEL`.
### Data directory (Gateway-owned)
Resolved inside the Gateway, not the Python provider:
1. `TDAI_DATA_DIR`
2. `data.baseDir` in `tdai-gateway.yaml` / `tdai-gateway.json`
3. Default `~/.memory-tencentdb/memory-tdai` (parent overridable via `MEMORY_TENCENTDB_ROOT`)
4. Legacy fallback `~/memory-tdai` with deprecation warning if new path missing
`MEMORY_TENCENTDB_DATA_DIR` is **not** read. Hermes inherits env into the child, so exporting `TDAI_DATA_DIR` before Hermes is sufficient.
### Minimal host `.env` example
```bash
# ~/.hermes/.env (systemd / load_dotenv)
MEMORY_TENCENTDB_GATEWAY_CMD="sh -c 'cd /home/you/.memory-tencentdb/tdai-memory-openclaw-plugin && exec /usr/bin/node --import tsx/esm src/gateway/server.ts'"
MEMORY_TENCENTDB_GATEWAY_HOST="127.0.0.1"
MEMORY_TENCENTDB_GATEWAY_PORT="8420"
MEMORY_TENCENTDB_LLM_API_KEY="sk-..."
MEMORY_TENCENTDB_LLM_BASE_URL="https://api.openai.com/v1"
MEMORY_TENCENTDB_LLM_MODEL="gpt-4o"
```
## Verify install
```bash
# 1. Hermes discovers the provider
cd
python -c 'from plugins.memory import discover_memory_providers; \
[print(n, a) for n, _, a in discover_memory_providers()]'
# expect: memory_tencentdb True
# 2. Gateway health
curl -s http://127.0.0.1:8420/health
# 3. Recall smoke (Docker or host)
curl -s -X POST http://127.0.0.1:8420/recall \
-H "Content-Type: application/json" \
-d '{"query":"test","session_key":"debug"}'
```
**Success signals**
| Signal | Where |
|---|---|
| `memory_tencentdb True` | `discover_memory_providers()` |
| Auto-discovery log | `memory-tencentdb Gateway command auto-discovered: …` in Hermes agent log |
| Gateway ready | `memory-tencentdb Gateway ready` / `already running` |
| Health | `status` is `ok` or `degraded` |
| Data tree | Under `TDAI_DATA_DIR` / `~/.memory-tencentdb/memory-tdai` |
## Troubleshooting
| Symptom | Check |
|---|---|
| Provider missing from discovery | Path is `…/plugins/memory/memory_tencentdb/` (underscore); `__init__.py` + `plugin.yaml` present |
| `memory-tencentdb Gateway not available` | CMD unset **and** discovery failed **and** nothing on `:8420`. Tail `~/.hermes/logs/memory_tencentdb/gateway.stderr.log` |
| Wrong checkout started | Set `MEMORY_TENCENTDB_GATEWAY_CMD` (always wins over discovery) |
| Tools missing from LLM | `get_tool_schemas()` returns `[]` until Gateway reachable **or** `MEMORY_TENCENTDB_GATEWAY_CMD` / `MEMORY_TENCENTDB_GATEWAY_PORT` set (optimistic register) |
| Circuit breaker tripped | Five failures; 60s pause; inspect Gateway health/logs |
| Capture backlog | ≥4 in-flight captures; look for stuck L1/LLM timeouts |
| systemd no auto-start | Put vars in `~/.hermes/.env`, not only `/etc/profile.d/` |
| Auth mismatches | Client key (`MEMORY_TENCENTDB_GATEWAY_API_KEY`) must match Gateway `TDAI_GATEWAY_API_KEY` / `server.apiKey` |
## Related pages
TdaiCore boundary and Hermes Gateway adapter path versus OpenClaw.
`/health`, `/recall`, `/capture`, search, `/session/end`, and auth fields.
`memory-tencentdb-ctl` start/stop/status and hermes vs standalone modes.
`MEMORY_TENCENTDB_ROOT` trees: conversations, records, vectors, persona.
SQLite vs tcvdb and embedding settings for the Gateway.
Gateway circuit breaker, silent memory, and recovery probes.
---
## 10. Seed historical conversations
> Run openclaw memory-tdai seed with Format A/B JSON input, config overrides, output directory layout, and L0→L1→L2→L3 verification signals.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/10-seed-historical-conversations.md
- Generated: 2026-07-09T07:33:24.031Z
### Source Files
- `src/cli/README.md`
- `src/cli/commands/seed.ts`
- `src/core/seed/input.ts`
- `src/core/seed/seed-runtime.ts`
- `src/core/seed/types.ts`
- `src/cli/index.ts`
---
title: "Seed historical conversations"
description: "Run openclaw memory-tdai seed with Format A/B JSON input, config overrides, output directory layout, and L0→L1→L2→L3 verification signals."
---
`openclaw memory-tdai seed` imports historical conversation JSON into a dedicated pipeline data directory: it validates Format A/B input, runs synchronous L0 capture via `performAutoCapture`, drains L1 on the `everyNConversations` boundary, wires L2/L3 runners from the shared pipeline factory, and writes a seed section into `.metadata/manifest.json`. The same `executeSeed` runtime powers Gateway `POST /seed`.
## Prerequisites
- OpenClaw plugin installed so the `memory-tdai` CLI namespace is registered (`api.registerCli` → `registerMemoryTdaiCli` → `registerSeedCommand`).
- LLM credentials available for L1 extraction (and L2/L3 if those stages run): either OpenClaw host config, or plugin `llm` settings used by `StandaloneLLMRunnerFactory` when `cfg.llm.enabled` and `cfg.llm.apiKey` are set.
- Optional embedding/store backend config if you need vector-backed L1 (sqlite-vec or tcvdb); defaults come from plugin config / `parseConfig`.
- Input file is non-empty JSON (Format A or B).
Seed writes to an **isolated output directory**, not necessarily the live `~/.openclaw/memory-tdai` tree. Point live runtime at that directory only if you intentionally want seeded data as the active store.
## Command
```bash
openclaw memory-tdai seed --input [options]
```
### Flags
| Flag | Required | Default | Behavior |
|------|----------|---------|----------|
| `--input ` | yes | — | Path to Format A/B JSON |
| `--output-dir ` | no | `/memory-tdai-seed-` | Pipeline data root (`stateDir` is typically `~/.openclaw`) |
| `--session-key ` | no | from input, else `"seed-user"` | Fallback when a session omits `sessionKey` during normalize |
| `--config ` | no | plugin config only | JSON deep-merged on top of current plugin config |
| `--strict-round-role` | no | `false` | Each round must include at least one `user` and one `assistant` message |
| `--yes` | no | `false` | Skip interactive timestamp auto-fill confirmation |
### Examples
```bash
openclaw memory-tdai seed --input conversations.json
openclaw memory-tdai seed --input data.json --output-dir ./seed-output --strict-round-role
openclaw memory-tdai seed --input data.json --config ./seed-config.json
openclaw memory-tdai seed --input data.json --yes
openclaw memory-tdai seed --input data.json --config seed-config.json --strict-round-role --yes
```
## Input formats
Both formats are detected in `loadAndValidateInput` / `extractSessions`.
### Format A — object wrapper
```json
{
"sessions": [
{
"sessionKey": "user-alice",
"sessionId": "conv-001",
"conversations": [
[
{ "role": "user", "content": "Hello", "timestamp": 1711929600000 },
{ "role": "assistant", "content": "Hi — how can I help?", "timestamp": 1711929601000 }
],
[
{ "role": "user", "content": "What is the weather today?" },
{ "role": "assistant", "content": "Clear and sunny." }
]
]
}
]
}
```
### Format B — top-level array
```json
[
{
"sessionKey": "user-alice",
"conversations": [
[
{ "role": "user", "content": "Hello" },
{ "role": "assistant", "content": "Hi!" }
]
]
}
]
```
### Session and message fields
| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `sessionKey` | string | yes (non-empty) | User/channel identity; multiple sessions may share a key with different `sessionId` |
| `sessionId` | string | no | Auto `crypto.randomUUID()` if omitted |
| `conversations` | `message[][]` | yes | Outer array = rounds; each inner array = messages in that round |
| `role` | string | yes | Non-empty string; typically `user` / `assistant` |
| `content` | string | yes | Non-empty after trim |
| `timestamp` | number \| string | no | Epoch **milliseconds** (integer) or ISO 8601 string |
## Validation layers
Seed runs a six-layer check before the pipeline starts:
| Layer | Stage id | What fails |
|-------|----------|------------|
| 1 | `file` | Missing file, empty file, JSON parse error |
| 2 | `top_level` | Neither Format A (`sessions` array) nor Format B (top-level array) |
| 3 | `session` | Empty sessions list; missing/empty `sessionKey`; `conversations` not an array |
| 4 | `round` | Round not an array; empty round; optional `--strict-round-role` role gaps |
| 5 | `message` | Missing `role`/`content`; bad timestamp type/format |
| 6 | `timestamp_consistency` | **Mixed** timestamps (some present, some missing) |
Timestamps are **all-or-none**. Mixed presence is a hard error. If **all** messages omit timestamps, CLI prompts to fill with current time (or auto-fills with `--yes`). Gateway `POST /seed` auto-fills by default (`auto_fill_timestamps`, default `true`).
Auto-fill uses a single monotonically increasing counter across **all** sessions (`Date.now()` then `+100` ms per message) so L0 capture cursors do not drop later sessions that share a `sessionKey`.
Validation failures throw `SeedValidationError` and exit the CLI with a multi-line summary (`[stage] session[i] round[j] msg[k] ...`).
## Config overrides
`--config` loads a JSON object and **two-level deep-merges** it onto the plugin config from OpenClaw:
- Top-level key is a plain object on both sides → shallow-merge children (`{ ...base, ...override }`).
- Otherwise → override wins.
Missing override file, invalid JSON, or non-object root → CLI exits with an error.
### Accelerate pipeline for bulk seed
```json
{
"pipeline": {
"everyNConversations": 3,
"enableWarmup": false,
"l1IdleTimeoutSeconds": 2,
"l2DelayAfterL1Seconds": 1,
"l2MinIntervalSeconds": 1,
"l2MaxIntervalSeconds": 10
}
}
```
Defaults without override (from `parseConfig`): `everyNConversations=5`, `enableWarmup=true`, `l1IdleTimeoutSeconds=600`, `l2DelayAfterL1Seconds=10`, `l2MinIntervalSeconds=900`, `l2MaxIntervalSeconds=3600`.
### Seed into an isolated TCVDB database
```json
{
"storeBackend": "tcvdb",
"tcvdb": {
"database": "my_seed_test_db"
},
"pipeline": {
"everyNConversations": 3,
"enableWarmup": false,
"l1IdleTimeoutSeconds": 2
}
}
```
## Pipeline behavior
```text
JSON input
│
▼
loadAndValidateInput / validateAndNormalizeRaw
│
▼
createSeedPipeline (createPipeline + L2/L3 runners)
│
▼
per session → per round
performAutoCapture (L0) pluginStartTimestamp = 0
every everyN rounds → waitForL1Idle
end of session → waitForL1Idle
final → waitForL1Idle (all session keys)
│
▼
pipeline.destroy()
manifest.seed written
SeedSummary printed / returned
```
### L0 capture
- Each round is mapped to `{ role, content, timestamp }` and passed to `performAutoCapture`.
- `pluginStartTimestamp` is forced to **`0`** so the live cold-start guard does not drop historical messages.
- L0 failures per round are logged; the loop continues.
### L1 batching and idle wait
- After every `everyNConversations` rounds **within a session**, seed pauses and polls `waitForL1Idle` so L1 batches align with the every-N boundary (mid-batch: poll 500 ms, 2 stable rounds, max 120 s; session tail / final: poll 1 s, 3 stable rounds, max 300 s).
- Idle means: scheduler reports `l1Idle`, buffered message count is 0, and session `conversation_count` is 0.
- If max wait elapses, seed logs a warning and proceeds.
### L2 / L3 (important constraint)
L2 (scene) and L3 (persona) runners are **wired** the same way as live runtime (`createL2Runner` / `createL3Runner`), but seed **only waits for L1 idle** before `pipeline.destroy()`. L2/L3 may still be in flight and can be interrupted. Summary counters report L0/L1-oriented stats (`l0RecordedCount`); they do not claim complete L2/L3 materialization.
For denser L2/L3 artifacts after seed, lower L2 interval delays via `--config`, keep the process alive longer after L1 drain when possible, and inspect `scene_blocks/` and `persona.md` rather than relying on the summary box alone.
### Interrupt handling
- First `SIGINT`: finish the current round, stop feeding new rounds, then destroy the pipeline.
- Second `SIGINT`: force `process.exit(1)`.
### Output directory guards
| Condition | Result |
|-----------|--------|
| `--output-dir` set | `path.resolve` that path |
| unset | `/memory-tdai-seed-` |
| dir exists + `.metadata/checkpoint.json` | Error: resume not implemented in P0 — use a new directory |
| dir exists, non-empty, no checkpoint | Error: clean or choose another directory |
| dir empty or missing | Proceed |
## Output directory layout
Created via `initDataDirectories` plus store init:
:::files
<output-dir>/
├── conversations/ # L0 JSONL (by day)
├── records/ # L1 JSONL
├── scene_blocks/ # L2 scene markdown (if L2 completes)
├── persona.md # L3 persona (if L3 completes; dataDir root)
├── vectors.db # sqlite backend only
├── .metadata/
│ ├── manifest.json # store binding + seed run record
│ └── checkpoint.json
└── .backup/
:::
### `manifest.json` seed section
On success, `executeSeed` appends:
```json
{
"version": 1,
"createdAt": "2026-04-01T22:00:00.000Z",
"store": {
"type": "sqlite",
"sqlite": { "path": "vectors.db" }
},
"seed": {
"inputFile": "conversations.json",
"sessions": 3,
"rounds": 42,
"messages": 128,
"startedAt": "2026-04-01T22:00:00.000Z",
"completedAt": "2026-04-01T22:05:30.000Z"
}
}
```
`inputFile` is stored as basename only. Manifest update failures are non-fatal.
## CLI progress and summary
During run:
```text
[12/42] 29% session=user-alice stage=l0_captured
[15/42] 36% session=user-alice stage=l1_waiting
```
On completion:
```text
╔══════════════════════════════════════════╗
║ Seed Summary ║
╠══════════════════════════════════════════╣
║ Sessions: 3 ║
║ Rounds: 42 ║
║ Messages: 128 ║
║ L0 recorded: 128 ║
║ Duration: 45.2s ║
╚══════════════════════════════════════════╝
📁 Output: /path/to/output-dir
```
## Verification signals (L0 → L1 → L2 → L3)
Use the seed **output directory** as the data root.
| Layer | Signal | How to check |
|-------|--------|--------------|
| **Run ok** | Summary box + logs | `Sessions` / `Rounds` / `L0 recorded` non-zero; logs contain `[memory-tdai] [seed] Seed complete` |
| **Manifest** | `seed` block present | Read `/.metadata/manifest.json` — `seed.sessions`, `seed.rounds`, `seed.messages`, timestamps |
| **L0** | Conversation JSONL | Files under `conversations/` (typically `YYYY-MM-DD.jsonl`) grow with seeded messages |
| **L1** | Memory records | Files under `records/`; vector rows in `vectors.db` when `storeBackend=sqlite` and embedding is enabled |
| **L2** | Scene blocks | Non-empty `scene_blocks/*.md` (may be incomplete if destroy cut L2 short) |
| **L3** | Persona | `persona.md` at output root (may be absent if L3 never finished or persona trigger threshold not reached) |
| **Search smoke** | Agent tools / Gateway | Point tools or Gateway data dir at the seed output, then call `tdai_memory_search` / `tdai_conversation_search` or Gateway search endpoints |
Minimum healthy seed: CLI exits 0, summary shows expected session/round counts, `conversations/` has content, `manifest.json` has a filled `seed` object, and `records/` (or vector store) shows L1 output after L1 idle wait.
## Gateway alternative: `POST /seed`
Standalone/Hermes Gateway reuses `validateAndNormalizeRaw` + `executeSeed`.
| Item | Value |
|------|--------|
| Method / path | `POST /seed` |
| Body | `SeedRequest`: `data` (required Format A/B), optional `session_key`, `strict_round_role`, `auto_fill_timestamps` (default `true`), `config_override` |
| Output dir | `/seed-` (not CLI `memory-tdai-seed-…`) |
| LLM | Gateway injects its `llm` block (`enabled: true` + baseUrl/apiKey/model/…) into plugin config before merge |
| Response | `sessions_processed`, `rounds_processed`, `messages_processed`, `l0_recorded`, `duration_ms`, `output_dir` |
| Validation errors | HTTP 400 with `error` + `validation_errors` |
Request is **blocking** and can take minutes for large inputs.
## Failure modes
| Symptom | Likely cause | Recovery |
|---------|--------------|----------|
| `Input file not found` / parse error | Bad `--input` path or JSON | Fix file path/content |
| `Timestamp consistency check failed` | Mixed timestamps | Supply timestamps for all messages or remove all |
| Prompt / abort on missing timestamps | Interactive confirm declined | Re-run with timestamps or `--yes` |
| `Resume from checkpoint is not implemented` | Output dir has `.metadata/checkpoint.json` | New `--output-dir` or remove old dir intentionally |
| `Output directory already exists and is not empty` | Stale files without checkpoint | Clean dir or choose another path |
| Config override not found / not object | Bad `--config` | Fix path; root must be a JSON object |
| L0 recorded but empty `records/` | L1 LLM failure, capture disabled, or interrupt | Check `[memory-tdai] [seed]` / L1 logs; verify `extraction.enabled` and LLM credentials |
| L1 present, no `scene_blocks/` / `persona.md` | L2/L3 not awaited or thresholds not met | Tune `pipeline` / `persona` via `--config`; re-seed or re-run live pipeline on the data dir |
| Second Ctrl+C force exit | User force quit | Treat as partial; re-seed into a **new** directory |
## Related pages
L0 conversation, L1 atom, L2 scene, and L3 persona model and storage shapes.
Full `openclaw memory-tdai seed` flag inventory and package bins.
`POST /seed` envelope, auth, and response fields for the Hermes sidecar.
On-disk trees for OpenClaw, context-offload, and Hermes roots.
Full plugin schema including `pipeline`, `llm`, embedding, and store backends used by seed overrides.
sqlite vs tcvdb and embedding settings for seed output stores.
`tdai_memory_search` and `tdai_conversation_search` for post-seed smoke checks.
Broader failure modes: silent plugin, embedding degrade, Gateway circuit breaker.
---
## 11. Migrate SQLite to Tencent VectorDB
> Build and run migrate-sqlite-to-tcvdb: dry-run, layer selection, openclaw.json rewrites, and post-migration storeBackend=tcvdb checks.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/11-migrate-sqlite-to-tencent-vectordb.md
- Generated: 2026-07-09T07:43:00.195Z
### Source Files
- `scripts/migrate-sqlite-to-tcvdb/README.md`
- `scripts/migrate-sqlite-to-tcvdb/cli-entry.ts`
- `scripts/migrate-sqlite-to-tcvdb/sqlite-to-tcvdb.ts`
- `scripts/migrate-sqlite-to-tcvdb/config-write.ts`
- `bin/migrate-sqlite-to-tcvdb.mjs`
- `SKILL-MIGRATION.md`
---
title: "Migrate SQLite to Tencent VectorDB"
description: "Build and run migrate-sqlite-to-tcvdb: dry-run, layer selection, openclaw.json rewrites, and post-migration storeBackend=tcvdb checks."
---
`migrate-sqlite-to-tcvdb` is the offline CLI that copies local memory data from SQLite (`vectors.db` via `VectorStore`) into Tencent Cloud VectorDB (`TcvdbMemoryStore`), then optionally rewrites OpenClaw plugin config to `storeBackend: "tcvdb"` and updates `/.metadata/manifest.json`. The package bin is `migrate-sqlite-to-tcvdb` (`bin/migrate-sqlite-to-tcvdb.mjs` → compiled `cli-entry.js`). Logs go to stderr under `[memory-tdai][migrate]` / `[memory-tdai][migrate-cli]`; the process prints a JSON migration summary on stdout.
This is **storage backend migration** (sqlite → tcvdb). It is not the package rename path from `@tdai/memory-tdai` to `@tencentdb-agent-memory/memory-tencentdb`. For package rename, see the package-rename docs page.
## Prerequisites
| Requirement | Detail |
|---|---|
| Node.js | `>= 22.16.0` (package `engines`) |
| Built migration artifacts | `npm run build:migrate-sqlite-to-vdb` (note the script name uses `vdb`) |
| Plugin data dir | Typically `~/.openclaw/memory-tdai` (shared data path; not deleted by plugin uninstall) |
| OpenClaw config | Single-file JSON/JSON5 path (default consumer: `~/.openclaw/openclaw.json`) |
| TCVDB endpoint | Reachable URL, username, database, embedding model, and API key |
| Embedding model | Must match the TCVDB collection / server-side embedding model you intend to use |
:::files
scripts/migrate-sqlite-to-tcvdb/
├── cli-entry.ts # CLI entry; prints JSON summary
├── sqlite-to-tcvdb.ts # parse args, preflight, migrate, verify
├── config-write.ts # openclaw.json storeBackend=tcvdb patch
├── manifest-write.ts # .metadata/manifest.json → type tcvdb
├── tsconfig.json
└── dist/ # tsc output (gitignored)
bin/migrate-sqlite-to-tcvdb.mjs
:::
## Build and invoke
Compile before the first run (or after script source changes):
```bash
npm run build:migrate-sqlite-to-vdb
```
Output lands under `scripts/migrate-sqlite-to-tcvdb/dist/` (entry: `dist/scripts/migrate-sqlite-to-tcvdb/cli-entry.js`).
Run via npm script, package bin, or node:
```bash title="npm script"
npm run migrate-sqlite-to-tcvdb -- \
--plugin-data-dir ~/.openclaw/memory-tdai \
--openclaw-config-path ~/.openclaw/openclaw.json \
--tcvdb-url http://127.0.0.1:80 \
--tcvdb-username root \
--tcvdb-api-key-env TCVDB_API_KEY \
--tcvdb-database agent_memory_prod \
--tcvdb-embedding-model bge-large-zh \
--dry-run
```
```bash title="package bin (after install)"
migrate-sqlite-to-tcvdb \
--plugin-data-dir ~/.openclaw/memory-tdai \
--openclaw-config-path ~/.openclaw/openclaw.json \
--tcvdb-url http://127.0.0.1:80 \
--tcvdb-username root \
--tcvdb-api-key-env TCVDB_API_KEY \
--tcvdb-database agent_memory_prod \
--tcvdb-embedding-model bge-large-zh \
--yes
```
`package.json` registers:
- **bin:** `migrate-sqlite-to-tcvdb`
- **scripts:** `build:migrate-sqlite-to-vdb`, `migrate-sqlite-to-tcvdb`
Use `npm run migrate-sqlite-to-tcvdb` (hyphenated). There is no `migrate:sqlite-to-tcvdb` script in `package.json`.
## Migration flow
```mermaid
flowchart LR
subgraph source [Local source]
SQLite["vectors.db
VectorStore L0/L1"]
Files["plugin-data-dir
scene_blocks + persona.md"]
Manifest[".metadata/manifest.json"]
end
subgraph cli [migrate-sqlite-to-tcvdb]
Preflight["collectMigrationPreflight"]
Migrate["page upsert L1/L0
syncProfiles"]
Verify["count verify + 10s delay"]
Config["writeMigrationPluginConfig"]
ManWrite["rewriteMigrationManifest"]
end
subgraph target [TCVDB + OpenClaw]
TCVDB["TcvdbMemoryStore"]
OC["openclaw.json
storeBackend=tcvdb"]
end
SQLite --> Preflight
Files --> Preflight
Manifest --> Preflight
Preflight -->|dry-run| Summary["JSON summary stdout"]
Preflight -->|live| Migrate
Migrate --> TCVDB
Migrate --> Verify
Verify --> Config
Config --> OC
Verify --> ManWrite
ManWrite --> Manifest
```
| Phase | Behavior |
|---|---|
| Preflight | Opens SQLite; counts L0/L1; lists local profiles; reads manifest store type |
| Dry-run | Returns preflight summary only; **no** data, config, or manifest writes |
| Data migrate | Paged upserts (page size **50**): L1 → L0 → profiles (when layers allow) |
| Target guard | Default: abort if target L0/L1/profile counts are non-zero |
| Verify | Default: wait **10s**, then require target counts to equal source counts |
| Config | Default: merge `storeBackend`, `tcvdb`, `bm25` into `plugins.entries..config` |
| Manifest | Default: set store binding to `type: "tcvdb"`; backup existing to `manifest.json.migrate.bak` |
**Empty / missing source handling**
- Missing `plugin-data-dir` or missing SQLite file → empty source counts, no throw (fresh install path).
- Source counts all zero → skip data migration; still apply config/manifest when not dry-run.
**What is not re-uploaded**
- Local sqlite-vec embedding blobs are not copied. `TcvdbMemoryStore` uses **server-side** embedding on upsert; migration passes records without client embeddings.
- On-disk JSONL conversation/record files stay in place; the CLI migrates SQLite-indexed L0/L1 rows and local profile files into TCVDB profile sync.
## CLI options
### Required
Plugin data directory (e.g. `~/.openclaw/memory-tdai`).
Path to a single-file OpenClaw config (JSON/JSON5). Writer fails if the file cannot be parsed as such.
TCVDB service URL.
TCVDB username.
TCVDB database name.
Server-side embedding model name (e.g. `bge-large-zh`).
API key — **exactly one** of:
| Flag | Role |
|---|---|
| `--tcvdb-api-key ` | Plaintext key |
| `--tcvdb-api-key-env ` | Read key from environment variable `VAR` |
Providing both throws. Missing both throws. Empty env value throws.
### Optional paths and identity
| Flag | Default | Notes |
|---|---|---|
| `--sqlite-path` | `/vectors.db` | Alternate snapshot DB |
| `--plugin-id` | `memory-tencentdb` | Key under `plugins.entries` when writing config |
| `--tcvdb-alias` | `""` | Optional alias stored in config + manifest |
| `--tcvdb-timeout-ms` | `10000` | Must be finite and `> 0` |
| `--tcvdb-ca-pem` | — | CA PEM path for HTTPS clients |
| `--summary-json-path` | — | Also write summary JSON to this file |
| `--job-id` | — | Accepted for tracing; stored on resolved options |
### Layer selection
| Flag | Default | Values |
|---|---|---|
| `--layers` | `l0,l1,l2,l3` | Comma-separated subset of `l0`, `l1`, `l2`, `l3` |
| Layer token | Migrates |
|---|---|
| `l1` | L1 memory rows from SQLite → `upsertL1` / `upsertL1Batch` |
| `l0` | L0 conversation rows from SQLite → `upsertL0` / `upsertL0Batch` |
| `l2` **or** `l3` | Local profiles via `listLocalProfiles` + `syncProfiles` |
Profile discovery (when `l2` or `l3` is selected):
- `scene_blocks/*.md` → profile type `l2`
- non-empty `persona.md` (navigation stripped) → profile type `l3`
Selecting only `l2` or only `l3` still runs the **same** profile sync (both L2 blocks and L3 persona if present). Omitting both skips profiles entirely.
### Boolean flags (defaults true unless noted)
Node `parseArgs` supports negation (`allowNegative: true`):
| Flag | Default | Effect |
|---|---|---|
| `--dry-run` | `false` | Preflight only; no writes |
| `--yes` | `false` | Recorded in summary; **no interactive prompt is implemented** — live runs proceed without a confirm gate |
| `--apply-config` / `--no-apply-config` | `true` | Write openclaw plugin config |
| `--config-backup` / `--no-config-backup` | `true` | Parsed into summary only; **config writer does not create an openclaw.json backup** |
| `--rewrite-manifest` / `--no-rewrite-manifest` | `true` | Rewrite data-dir manifest to `tcvdb` |
| `--fail-if-target-nonempty` / `--no-fail-if-target-nonempty` | `true` | Abort when target already has L0/L1/profiles |
| `--verify-counts` / `--no-verify-counts` | `true` | Post-migrate exact count match (with 10s settle delay) |
| `--bm25-enabled` / `--no-bm25-enabled` | `true` | BM25 encoder for target store + config patch |
| `--bm25-language` | `zh` | `zh` or `en` only |
Back up `openclaw.json` yourself before a live run. Manifest backup is automatic (`manifest.json.migrate.bak`); openclaw config backup is not performed by the writer.
## openclaw.json rewrite
When `--apply-config` is enabled (default), `writeMigrationPluginConfig` merges into:
```text
plugins.entries..config
```
Default `pluginId` is `memory-tencentdb`. Patch shape:
```json
{
"storeBackend": "tcvdb",
"tcvdb": {
"url": "",
"username": "",
"apiKey": "",
"database": "",
"alias": "",
"embeddingModel": "",
"timeout": 10000
},
"bm25": {
"enabled": true,
"language": "zh"
}
}
```
Merge rules:
- Existing entry and nested `tcvdb` / `bm25` objects are shallow-merged (patch wins on overlapping keys).
- Other plugin config keys (embedding, extraction, capture, etc.) are preserved.
- Input is parsed with **JSON5**; output is pretty-printed JSON with a trailing newline.
- Writer requires a single-file config; multi-file layouts are not supported.
Disable automatic rewrite when you manage config out of band:
```bash
npm run migrate-sqlite-to-tcvdb -- \
...required flags... \
--no-apply-config \
--no-rewrite-manifest \
--yes
```
## Manifest rewrite
When `--rewrite-manifest` is enabled (default):
| Case | Result |
|---|---|
| No existing manifest | Creates version `1` with `store.type: "tcvdb"` |
| Existing manifest | Copies current file to `.metadata/manifest.json.migrate.bak`, then updates `store` only |
Store info includes `url`, `database`, and optional `alias`. Relative sqlite path is not retained after rewrite.
## Command recipes
### Dry-run preflight
```bash
export TCVDB_API_KEY='***'
npm run migrate-sqlite-to-tcvdb -- \
--plugin-data-dir ~/.openclaw/memory-tdai \
--openclaw-config-path ~/.openclaw/openclaw.json \
--tcvdb-url http://127.0.0.1:80 \
--tcvdb-username root \
--tcvdb-api-key-env TCVDB_API_KEY \
--tcvdb-database agent_memory_prod \
--tcvdb-embedding-model bge-large-zh \
--dry-run
```
Stdout is a JSON `MigrationPreflightSummary` with `source.l0Count` / `l1Count` / `profileCount`, `manifestStoreType`, target connection fields, and option flags. No `migration` block is attached on dry-run.
### Full live migration
```bash
cp ~/.openclaw/openclaw.json ~/.openclaw/openclaw.json.bak
npm run migrate-sqlite-to-tcvdb -- \
--plugin-data-dir ~/.openclaw/memory-tdai \
--openclaw-config-path ~/.openclaw/openclaw.json \
--tcvdb-url http://127.0.0.1:80 \
--tcvdb-username root \
--tcvdb-api-key-env TCVDB_API_KEY \
--tcvdb-database agent_memory_prod \
--tcvdb-embedding-model bge-large-zh \
--yes
```
On success, summary includes `migration` with migrated counts, target counts, `configWritten`, `manifestWritten`, and optional `manifestBackupPath`.
### L1 only
```bash
# ...required connection flags...
--layers l1 \
--yes
```
### L0 + L1 without profiles
```bash
--layers l0,l1 \
--yes
```
### Custom SQLite snapshot
```bash
--sqlite-path /backup/2026-04/vectors-snapshot.db \
--yes
```
### English BM25
```bash
--bm25-language en \
--tcvdb-embedding-model bge-large-en-v1.5 \
--yes
```
### Dense-only (no BM25 sparse)
```bash
--no-bm25-enabled \
--yes
```
### Append into non-empty target (unsafe for strict cutover)
```bash
--no-fail-if-target-nonempty \
--no-verify-counts \
--yes
```
Count verification is usually disabled together with non-empty target, because target totals will not equal source-only counts.
### CI summary artifact
```bash
--summary-json-path ./migration-report.json \
--job-id migrate-2026-04-13 \
--yes
```
## Post-migration checks
After a live run with default flags:
Inspect the plugin entry (default id `memory-tencentdb`):
```bash
python3 - <<'PY'
import json, pathlib
p = pathlib.Path.home() / ".openclaw" / "openclaw.json"
cfg = json.loads(p.read_text())
entry = cfg.get("plugins", {}).get("entries", {}).get("memory-tencentdb", {})
c = entry.get("config", entry)
print("storeBackend =", c.get("storeBackend"))
print("tcvdb.database =", (c.get("tcvdb") or {}).get("database"))
print("bm25 =", c.get("bm25"))
assert c.get("storeBackend") == "tcvdb"
assert (c.get("tcvdb") or {}).get("url")
assert (c.get("tcvdb") or {}).get("apiKey")
assert (c.get("tcvdb") or {}).get("database")
print("ok")
PY
```
Runtime factory requires `tcvdb.url`, `tcvdb.apiKey`, and `tcvdb.database` when `storeBackend === "tcvdb"`.
```bash
cat ~/.openclaw/memory-tdai/.metadata/manifest.json
# expect store.type == "tcvdb"
# optional: .metadata/manifest.json.migrate.bak from pre-migration sqlite binding
```
```bash
openclaw gateway restart
# Expect store logs with backend=tcvdb (factory tag [memory-tdai][factory])
# Exercise tdai_memory_search / conversation search or a short dialogue
```
Use the printed `migration` object:
- `l1Migrated` / `targetL1Count` vs preflight `source.l1Count`
- `l0Migrated` / `targetL0Count` vs `source.l0Count`
- `profileMigrated` / `targetProfileCount` vs `source.profileCount`
With default `--verify-counts`, a mismatch already failed the process.
Success signals: process exit 0, stdout JSON with `migration.configWritten: true` (unless disabled), `storeBackend: "tcvdb"` in OpenClaw plugin config, manifest `store.type: "tcvdb"`, Gateway loading without TCVDB init errors.
## Failure modes
| Symptom | Likely cause | Recovery |
|---|---|---|
| `Missing required option --...` | Incomplete argv | Pass all required flags; API key via key **or** env |
| `Provide either --tcvdb-api-key or --tcvdb-api-key-env, not both` | Both key inputs set | Keep one form only |
| `Environment variable X is empty or not set` | Bad `--tcvdb-api-key-env` | Export the variable before run |
| `Failed to open sqlite store...` / degraded source | Corrupt or unreadable `vectors.db` | Repair/restore sqlite; re-run dry-run |
| `Target store entered degraded mode...` | TCVDB init/network/auth failure | Fix URL/credentials/network/CA PEM |
| `Target store is not empty (L1=..., L0=..., profiles=...)` | Default nonempty guard | Empty target DB, change `--tcvdb-database`, or `--no-fail-if-target-nonempty` |
| `L1/L0/Profile count verification failed` | Partial write or settle lag | Inspect TCVDB counts; re-run after clean target; avoid disabling verify until root cause is known |
| `Failed to batch migrate L1/L0...` | Batch upsert returned 0 | Check TCVDB write quotas/errors; fix then re-run on empty target |
| `Config migration writer only supports single-file JSON/JSON5` | Unreadable path or non-JSON5 content | Point `--openclaw-config-path` at a valid file |
| `Unsupported layer(s)` / `Unsupported --bm25-language` | Bad enum value | Use `l0,l1,l2,l3` and `zh`/`en` only |
| Bin import fails | Dist not built | Run `npm run build:migrate-sqlite-to-vdb` |
Exit behavior: uncaught errors set `process.exitCode = 1` and print `[memory-tdai][migrate-cli] ` on stderr.
## Operational notes
- **Idempotency:** Default path is a **cutover** into an empty TCVDB database, not a merge. Upserts may overwrite by id if you force nonempty targets, but count verification will fail unless disabled.
- **Secrets:** Prefer `--tcvdb-api-key-env`. Live config rewrite writes `apiKey` into `openclaw.json` in cleartext — protect file permissions and backups.
- **Data dir:** SQLite file and local JSONL trees remain after migration; runtime switches via `storeBackend`, not by deleting local files.
- **Plugin id vs log tag:** Config key defaults to `memory-tencentdb`; many runtime log prefixes still say `memory-tdai`.
- **Does not depend on `openclaw/plugin-sdk`:** migration modules import core store code and `json5` only.
## Related pages
Choose sqlite vs tcvdb, required tcvdb fields, embedding, and BM25 language.
Full plugin schema for `storeBackend`, `tcvdb`, `bm25`, and related keys.
Package bins including `migrate-sqlite-to-tcvdb` and build scripts.
On-disk layout for `~/.openclaw/memory-tdai`, vectors.db, persona, and scene_blocks.
Package rename workflow (not storage backend migration).
Runtime failure modes after store switches (embedding, recall, gateway).
---
## 12. Migrate from memory-tdai package
> Rename path from @tdai/memory-tdai to @tencentdb-agent-memory/memory-tencentdb: backup openclaw.json, uninstall old plugin, preserve ~/.openclaw/memory-tdai data, reinstall and verify.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/12-migrate-from-memory-tdai-package.md
- Generated: 2026-07-09T07:33:55.629Z
### Source Files
- `SKILL-MIGRATION.md`
- `package.json`
- `openclaw.plugin.json`
- `index.ts`
- `CHANGELOG.md`
---
title: "Migrate from memory-tdai package"
description: "Rename path from @tdai/memory-tdai to @tencentdb-agent-memory/memory-tencentdb: backup openclaw.json, uninstall old plugin, preserve ~/.openclaw/memory-tdai data, reinstall and verify."
---
Package rename migration for OpenClaw installs: replace npm package `@tdai/memory-tdai` (plugin id `memory-tdai`) with `@tencentdb-agent-memory/memory-tencentdb` (plugin id `memory-tencentdb`). Runtime data stays under `/memory-tdai` (default `~/.openclaw/memory-tdai`). Uninstall removes the plugin config entry and **does not** delete that data directory. Log tag, CLI namespace, and tool names remain `memory-tdai` / `tdai_*` for compatibility.
## What changes and what stays
| Surface | Old | New / unchanged |
|---|---|---|
| npm package | `@tdai/memory-tdai` | `@tencentdb-agent-memory/memory-tencentdb` |
| Plugin id | `memory-tdai` | `memory-tencentdb` |
| `openclaw.json` entry key | `plugins.entries.memory-tdai` | `plugins.entries.memory-tencentdb` |
| Data directory | `~/.openclaw/memory-tdai/` | **Unchanged** (hard-coded basename `memory-tdai` under OpenClaw state dir) |
| Log prefix | `[memory-tdai]` | **Unchanged** |
| CLI namespace | `openclaw memory-tdai …` | **Unchanged** (`commandAliases: ["memory-tdai"]`) |
| Agent tools | `tdai_memory_search`, `tdai_conversation_search` | **Unchanged** |
| Context Engine slot | must match plugin id | `plugins.slots.contextEngine: "memory-tencentdb"` |
| Current package version (repo) | — | `0.3.6` in `package.json` |
```text
Uninstall old plugin Install new package
───────────────────── ──────────────────
openclaw plugins uninstall openclaw plugins install
memory-tdai @tencentdb-agent-memory/
memory-tencentdb
│ │
│ removes plugins.entries.memory-tdai │
│ ▼
│ plugins.entries.memory-tencentdb
│ │
▼ ▼
~/.openclaw/memory-tdai/ ────────────────────► same paths
conversations/ records/ scene_blocks/
vectors.db persona.md (.metadata / .backup)
```
Uninstalling `memory-tdai` **deletes** its `openclaw.json` configuration segment (including `embedding.apiKey`, models, retention). Always backup before uninstall. The on-disk memory tree is not removed by uninstall.
## When to use this migration
**Use when:**
- `openclaw plugins list` shows `memory-tdai` / `@tdai/memory-tdai` loaded
- `openclaw plugins install @tdai/memory-tdai` fails with 404 / not found
- You were told the old package name is retired
**Do not use when:**
- No memory plugin is installed yet — use a fresh install of `@tencentdb-agent-memory/memory-tencentdb` instead
- You only need SQLite → Tencent VectorDB data migration (separate tool `migrate-sqlite-to-tcvdb`)
### Prerequisites
| Requirement | Constraint |
|---|---|
| Node.js | `>= 22.16.0` (`package.json` `engines`) |
| OpenClaw plugin API / gateway | `>= 2026.3.13` (`package.json` `openclaw.compat`) |
| Peer OpenClaw (optional peer) | `>= 2026.3.7` |
```bash
node -v
openclaw --version
openclaw plugins list | grep -i memory
```
## Identity map (config and runtime)
| Identifier | Value | Notes |
|---|---|---|
| New package name | `@tencentdb-agent-memory/memory-tencentdb` | npm / `openclaw plugins install` |
| New plugin id | `memory-tencentdb` | `openclaw.plugin.json` `"id"`; config key under `plugins.entries` |
| Old plugin id | `memory-tdai` | Uninstall target; old config key |
| CLI alias | `memory-tdai` | Still registered; seed commands stay `openclaw memory-tdai seed …` |
| Log tag | `[memory-tdai]` | Expected after successful migration |
| Data basename | `memory-tdai` | `path.join(openclawStateDir, "memory-tdai")` |
| State dir resolution | `runtime.state.resolveStateDir()` → `OPENCLAW_STATE_DIR` → `~/.openclaw` | |
Minimal enable shape after migration (flat form also used in README):
```json
{
"memory-tencentdb": {
"enabled": true
}
}
```
OpenClaw host config used by uninstall/restore workflow is typically nested as:
```json
{
"plugins": {
"entries": {
"memory-tencentdb": {
"enabled": true,
"config": { }
}
}
}
}
```
Restore whatever structure you backed up under the **new** plugin id key. Preserve nested `config` fields (`embedding`, `extraction`, `persona`, `capture`, `pipeline`, `recall`, `offload`, `storeBackend`, `tcvdb`, …) exactly.
## Migration procedure
```bash
openclaw plugins list | grep -i memory
```
Expect `memory-tdai` or `@tdai/memory-tdai` loaded. If nothing matches, skip this page and install `@tencentdb-agent-memory/memory-tencentdb` as a new install.
Uninstall removes the old entry. Extract and save it:
```bash
cat ~/.openclaw/openclaw.json | python3 -c "
import sys, json
cfg = json.load(sys.stdin)
plugins = cfg.get('plugins', {}).get('entries', {})
old_cfg = plugins.get('memory-tdai', {})
if old_cfg:
print(json.dumps(old_cfg, indent=2, ensure_ascii=False))
with open('/tmp/memory-tdai-config-backup.json', 'w') as f:
json.dump(old_cfg, f, indent=2, ensure_ascii=False)
print('\nConfig backed up to /tmp/memory-tdai-config-backup.json')
else:
print('No memory-tdai entry found (may already use defaults only)')
"
```
Record these if present (restore failures usually start here):
| Config path | Why it matters |
|---|---|
| `embedding` (`provider`, `baseUrl`, `apiKey`, `model`, `dimensions`, `proxyUrl`, `sendDimensions`) | Incomplete quadruplet degrades vector search |
| `extraction.model` / `persona.model` | L1/L2/L3 model routing |
| `capture.excludeAgents` | Agent capture filter |
| `capture.l0l1RetentionDays` / `allowAggressiveCleanup` | Cleanup aggressiveness |
| `offload` + `plugins.slots.contextEngine` | Offload enable + slot id |
| `storeBackend` / `tcvdb` / `bm25` | Backend choice and remote VDB credentials |
`/tmp/memory-tdai-config-backup.json` may contain plaintext `apiKey` values. Restrict access; delete after a successful migration.
```bash
ls -la ~/.openclaw/memory-tdai/
```
Typical layout after use:
```text
~/.openclaw/memory-tdai/
├── conversations/ # L0 JSONL
├── records/ # L1 atoms
├── scene_blocks/ # L2 scenes
├── vectors.db # sqlite-vec store (if storeBackend=sqlite)
├── persona.md # L3 persona
├── .metadata/
└── .backup/
```
Capture pre-migration counts:
```bash
echo "=== pre-migration stats ==="
wc -l ~/.openclaw/memory-tdai/conversations/*.jsonl 2>/dev/null || echo "no conversations"
wc -l ~/.openclaw/memory-tdai/records/*.jsonl 2>/dev/null || echo "no records"
ls ~/.openclaw/memory-tdai/scene_blocks/*.md 2>/dev/null | wc -l | xargs -I{} echo "scene blocks: {}"
wc -c ~/.openclaw/memory-tdai/persona.md 2>/dev/null || echo "no persona"
```
If `OPENCLAW_STATE_DIR` is set, the data root is `$OPENCLAW_STATE_DIR/memory-tdai` instead of `~/.openclaw/memory-tdai`.
```bash
openclaw plugins uninstall memory-tdai
```
Verify:
| Check | Expected |
|---|---|
| `plugins.entries.memory-tdai` | Gone |
| `~/.openclaw/memory-tdai/` | Still present |
```bash
ls ~/.openclaw/memory-tdai/ && echo "data dir intact" || echo "data dir missing"
```
```bash
openclaw plugins install @tencentdb-agent-memory/memory-tencentdb
```
For later updates, prefer:
```bash
openclaw plugins update @tencentdb-agent-memory/memory-tencentdb
```
Native OpenClaw install/update avoids disabling the plugin from loose semantic version ranges. Package `postinstall` runs `scripts/openclaw-after-tool-call-messages.patch.sh` (best-effort; failures are ignored).
Write the backup to the **new** plugin id:
```bash
python3 -c "
import json, os
backup_path = '/tmp/memory-tdai-config-backup.json'
if os.path.exists(backup_path):
with open(backup_path) as f:
old_cfg = json.load(f)
print('Restoring backup:')
print(json.dumps(old_cfg, indent=2, ensure_ascii=False))
else:
old_cfg = {'enabled': True}
print('No backup found; writing minimal enabled config')
config_path = os.path.expanduser('~/.openclaw/openclaw.json')
with open(config_path) as f:
cfg = json.load(f)
cfg.setdefault('plugins', {}).setdefault('entries', {})['memory-tencentdb'] = old_cfg
with open(config_path, 'w') as f:
json.dump(cfg, f, indent=2, ensure_ascii=False)
print('Wrote plugins.entries.memory-tencentdb')
"
```
If the backup is missing, at least:
```json
{
"plugins": {
"entries": {
"memory-tencentdb": {
"enabled": true
}
}
}
}
```
If you use context offload, ensure the slot points at the new id (not an old engine id):
```json
{
"plugins": {
"slots": {
"contextEngine": "memory-tencentdb"
}
}
}
```
`registerContextEngine` / `setup-offload.sh` use `CONTEXT_ENGINE_ID=memory-tencentdb` so `openclaw doctor --fix` does not reset the slot against a mismatched plugin name.
```bash
openclaw gateway restart
```
| Signal | How to check | Pass criteria |
|---|---|---|
| Plugin loaded | `openclaw plugins list \| grep -i memory` | `memory-tencentdb` present |
| Runtime log tag | Gateway logs | Lines with `[memory-tdai]` (legacy tag is normal) |
| Data preserved | `wc -l` / `ls` under data dir | Matches pre-migration stats |
| Config present | `openclaw.json` | `plugins.entries.memory-tencentdb` (or flat `memory-tencentdb`) with `enabled: true` and restored nested settings |
```bash
echo "=== post-migration verify ==="
openclaw plugins list | grep -i memory
wc -l ~/.openclaw/memory-tdai/conversations/*.jsonl 2>/dev/null
wc -l ~/.openclaw/memory-tdai/records/*.jsonl 2>/dev/null
```
1. Send a turn that contains durable personal facts (preference, constraint, background).
2. Confirm Gateway logs show `[before_prompt_build]` / `[agent_end]` activity under `[memory-tdai]`.
3. Optionally call tools `tdai_memory_search` and `tdai_conversation_search`.
4. If embedding was configured, confirm no `[EMBEDDING CONFIG ERROR]` / incomplete-config degrade messages.
```bash
rm -f /tmp/memory-tdai-config-backup.json
```
## Compatible surfaces that keep old names
These intentionally retain `memory-tdai` branding so scripts and data stay stable:
| Surface | Identifier | Source behavior |
|---|---|---|
| On-disk plugin data | `…/memory-tdai` | `path.join(openclawStateDir, "memory-tdai")` |
| Log tag | `[memory-tdai]` | `const TAG = "[memory-tdai]"` in plugin entry |
| CLI | `openclaw memory-tdai seed …` | `commandAliases: ["memory-tdai"]` + CLI registration |
| Tools | `tdai_memory_search`, `tdai_conversation_search` | `openclaw.plugin.json` `contracts.tools` |
| SQLite migrate default data path | `--plugin-data-dir ~/.openclaw/memory-tdai` | migrate tool docs; `--plugin-id` default is already `memory-tencentdb` |
| Hermes / standalone data (non-OpenClaw) | `$MEMORY_TENCENTDB_ROOT/memory-tdai` (default `~/.memory-tencentdb/memory-tdai`) | Gateway config defaults; separate from OpenClaw plugin data |
Seeing `[memory-tdai]` in logs after installing `memory-tencentdb` is expected, not a failed rename. Plugin **id** and **npm name** changed; log tag and data basename did not.
## Rollback
If post-migration behavior is wrong and the old package is still reachable on npm:
```bash
openclaw plugins uninstall memory-tencentdb
openclaw plugins install @tdai/memory-tdai
# Restore /tmp/memory-tdai-config-backup.json under plugins.entries.memory-tdai
openclaw gateway restart
```
If the old package 404s, keep the new package installed, re-apply config from backup, and fix config keys rather than reinstalling the old id. Data under `memory-tdai/` remains the source of truth either way.
## Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| No plugin logs after restart | `memory-tencentdb.enabled` not true / wrong config key | Ensure `plugins.entries.memory-tencentdb` (or documented enable shape) has `enabled: true`; restart gateway |
| Install of old package 404 | Package renamed | Install `@tencentdb-agent-memory/memory-tencentdb` only |
| Install of new package fails | Registry / network | Check npm registry and connectivity |
| History missing after migrate | Config not restored; recall disabled; wrong data dir | Diff backup vs live config; confirm data path still has files; check `recall.enabled` |
| Embedding errors / keyword-only | Lost `apiKey` / `baseUrl` / `model` / `dimensions` | Restore full `embedding` block from backup |
| Empty data directory | Unexpected delete (rare) or wrong `OPENCLAW_STATE_DIR` | Locate actual state dir; do not recreate empty tree over a moved path |
| Offload / context engine inactive | Slot still points at old engine id | Set `plugins.slots.contextEngine` to `"memory-tencentdb"`; re-run after-tool-call patch if needed |
| `migrate-sqlite-to-tcvdb` writes wrong plugin id | Explicit old `--plugin-id` | Default is `memory-tencentdb`; pass that id when rewriting `openclaw.json` |
## Security notes
- Treat backup JSON as secret material (`embedding.apiKey`, `tcvdb.apiKey`, `llm.apiKey`, offload backend keys).
- Prefer not to paste full configs into chat; redact keys in support exports.
- Limit edits to the memory plugin entry and related slots; do not rewrite unrelated `plugins.entries`.
- Delete `/tmp/memory-tdai-config-backup.json` after verification.
## Definition of done
Migration is complete only when all of the following hold:
- [ ] `@tdai/memory-tdai` / plugin id `memory-tdai` uninstalled
- [ ] `@tencentdb-agent-memory/memory-tencentdb` installed and listed as loaded
- [ ] `openclaw.json` has full `memory-tencentdb` config (custom embedding and other overrides restored)
- [ ] Gateway restarted
- [ ] Logs show `[memory-tdai]`
- [ ] `~/.openclaw/memory-tdai/` (or `$OPENCLAW_STATE_DIR/memory-tdai`) counts match pre-migration inventory
- [ ] At least one conversation turn proves capture/recall path
- [ ] Sensitive backup file removed
## Related pages
Fresh install, update, link-from-source, and postinstall patch behavior for the current package.
Zero-config enable, gateway restart, log and data-dir success signals.
Layout of `memory-tdai`, context-offload, and Hermes `MEMORY_TENCENTDB_ROOT` trees.
Backend migration tool; uses data dir `memory-tdai` and default plugin id `memory-tencentdb`.
`offload.enabled`, `plugins.slots.contextEngine: memory-tencentdb`, and after-tool-call patch.
Full plugin config schema restored during migration (embedding, capture, pipeline, offload, …).
Silent plugin, missing recall, embedding degrade, and recovery probes.
---
## 13. Configuration reference
> Full plugin config schema: timezone, storeBackend, capture, extraction, persona, pipeline, recall, embedding, tcvdb, bm25, report, llm, offload — defaults, constraints, and degrade rules.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/13-configuration-reference.md
- Generated: 2026-07-09T07:34:30.468Z
### Source Files
- `src/config.ts`
- `openclaw.plugin.json`
- `src/gateway/config.ts`
- `src/utils/no-think-fetch.ts`
- `src/core/store/embedding.ts`
- `SKILL.md`
---
title: "Configuration reference"
description: "Full plugin config schema: timezone, storeBackend, capture, extraction, persona, pipeline, recall, embedding, tcvdb, bm25, report, llm, offload — defaults, constraints, and degrade rules."
---
Plugin configuration is parsed by `parseConfig()` into a fully resolved `MemoryTdaiConfig`. Minimal valid config is `{}` — every field has a runtime default. OpenClaw hosts place the object under `memory-tencentdb` in `~/.openclaw/openclaw.json`. The Gateway reuses the same `memory` object shape nested under `tdai-gateway.yaml` / `tdai-gateway.json`. The JSON Schema in `openclaw.plugin.json` drives the OpenClaw UI; runtime behavior always follows `src/config.ts`.
```json
{
"memory-tencentdb": {
"enabled": true
}
}
```
## Config surfaces
| Surface | Where config lives | Parser |
| :--- | :--- | :--- |
| OpenClaw plugin | `~/.openclaw/openclaw.json` → `memory-tencentdb` | `parseConfig(api.pluginConfig)` in plugin `register()` |
| Standalone / Hermes Gateway | `tdai-gateway.yaml` / `.json` → `memory` (+ top-level `server`, `data`, `llm`) | `loadGatewayConfig()` then `parseConfig(memory)` |
| Seed CLI | `--config` / plugin config overrides | `parseConfig(pluginConfig)` |
Env overrides apply to **Gateway server/LLM/data only** (for example `TDAI_GATEWAY_PORT`, `TDAI_LLM_API_KEY`). Memory group fields are not individual env vars; they come from the file’s `memory` object.
```text
openclaw.json / tdai-gateway.yaml
│
▼
parseConfig() ──► MemoryTdaiConfig
│
├── capture → memoryCleanup (derived)
├── embedding ──► configError + enabled=false on incomplete remote
├── storeBackend + tcvdb ──► createStoreBundle()
├── llm ──► host LLM vs StandaloneLLMRunner
└── offload ──► context engine slot / local|backend|collect
```
## Top-level fields
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `timezone` | `string` | `"system"` | `"system"`, IANA name (`Asia/Shanghai`), or UTC offset (`+08:00`). Storage timestamps stay UTC; this only affects display and local-day cleanup boundaries. |
| `storeBackend` | `"sqlite" \| "tcvdb"` | `"sqlite"` | Any value other than `"tcvdb"` resolves to `"sqlite"`. |
| `capture` | object | see below | L0 capture + retention switch |
| `extraction` | object | see below | L1 extraction |
| `persona` | object | see below | L2 scenes + L3 persona |
| `pipeline` | object | see below | L1→L2→L3 scheduling |
| `recall` | object | see below | Auto-recall injection |
| `embedding` | object | see below | Client-side embedding (sqlite path) |
| `tcvdb` | object | see below | Required when `storeBackend=tcvdb` |
| `bm25` | object | see below | Sparse encoder (esp. tcvdb hybrid) |
| `report` | object | see below | Metric JSON logs |
| `llm` | object | see below | Standalone OpenAI-compatible LLM for L1/L2/L3 |
| `offload` | object | see below | Context Offload (independent of long-term memory) |
`memoryCleanup` is **not** a user-facing top-level key. It is derived from `capture.l0l1RetentionDays`, `capture.allowAggressiveCleanup`, and `capture.cleanTime`.
---
## `timezone`
| Property | Type | Default | Constraints |
| :--- | :--- | :--- | :--- |
| `timezone` | `string` | `"system"` | Follows process TZ when `"system"`; otherwise IANA or ECMA-402 offset |
Passed to `initTimeModule({ timezone })` before any timestamp formatting or day-boundary cleanup.
---
## `storeBackend`
| Value | Behavior |
| :--- | :--- |
| `sqlite` (default) | Local `vectors.db` (sqlite-vec + FTS5). Client embedding via `embedding.*`. |
| `tcvdb` | Tencent Cloud VectorDB. Server-side embedding; client uses `NoopEmbeddingService`. Requires `tcvdb.url`, `tcvdb.apiKey`, and `tcvdb.database`. |
Missing tcvdb required fields throw at `createStoreBundle()`; store init failure wires the pipeline in **degraded mode** (JSONL capture may still work; vector recall/dedup is unavailable).
---
## `capture` (L0)
| Key | Type | Default | Constraints / notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `true` | Disable stops auto-capture |
| `excludeAgents` | `string[]` | `[]` | Glob patterns (e.g. `bench-judge-*`). Matched agents skip capture, recall, and pipeline scheduling |
| `l0l1RetentionDays` | `number` | `0` | `0` = no cleanup. Effective retention only when `>= 3`, or `1`/`2` with `allowAggressiveCleanup: true` |
| `allowAggressiveCleanup` | `boolean` | `false` | Required for retention of 1–2 days |
| `cleanTime` | `string` | `"03:00"` | `HH:mm` or `H:mm` (24h). Invalid values fall back to `"03:00"` |
### Retention degrade rules
| Input `l0l1RetentionDays` | `allowAggressiveCleanup` | Effective result |
| :--- | :--- | :--- |
| `<= 0` | any | Cleanup disabled (`retentionDays` undefined, `l0l1RetentionDays` stored as `0`) |
| `>= 3` | any | Cleanup enabled with that TTL |
| `1` or `2` | `true` | Cleanup enabled (aggressive) |
| `1` or `2` | `false` | **Silently forced to disabled** (same as `0`) |
`cleanTime` normalization: hour 0–23, minute 0–59, minutes must be two digits (`3:5` is invalid).
---
## `extraction` (L1)
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `true` | Background L1 extraction |
| `enableDedup` | `boolean` | `true` | Vector/keyword conflict detection on L1 write |
| `maxMemoriesPerSession` | `number` | `20` | Cap per L1 pass |
| `model` | `string` | omit | `provider/model`; omit uses host/default model (or `llm.*` when standalone LLM is enabled) |
---
## `persona` (L2 / L3)
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `triggerEveryN` | `number` | `50` | Persona rebuild every N new L1 memories |
| `maxScenes` | `number` | `15` | Max scene blocks |
| `backupCount` | `number` | `3` | Persona file backup count |
| `sceneBackupCount` | `number` | `10` | Scene block backup count |
| `model` | `string` | omit | `provider/model`; omit uses host/default |
---
## `pipeline` (L1→L2→L3 scheduling)
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `everyNConversations` | `number` | `5` | Steady-state L1 trigger period (conversation rounds) |
| `enableWarmup` | `boolean` | `true` | New session: threshold starts at 1 and doubles after each L1 (`1→2→4→…→everyN`) |
| `l1IdleTimeoutSeconds` | `number` | `600` | L1 after user idle |
| `l2DelayAfterL1Seconds` | `number` | `10` | Delay after L1 before L2 |
| `l2MinIntervalSeconds` | `number` | `900` | Min seconds between L2 runs per session |
| `l2MaxIntervalSeconds` | `number` | `3600` | Max L2 poll interval for an active session |
| `sessionActiveWindowHours` | `number` | `24` | Sessions idle longer stop L2 polling |
---
## `recall`
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `true` | Auto-recall injection |
| `maxResults` | `number` | `5` | Max injected L1 hits |
| `maxCharsPerMemory` | `number` | `0` | Per-memory char cap; `0` = unlimited |
| `maxTotalRecallChars` | `number` | `0` | Total auto-recall budget; `0` = unlimited |
| `scoreThreshold` | `number` | `0.3` | Minimum score |
| `strategy` | `string` | `"hybrid"` | `"keyword"` \| `"embedding"` \| `"hybrid"` |
| `timeoutMs` | `number` | `5000` | Overall recall budget; on timeout, skip injection and log a warning |
### Strategy and degrade
| Configured strategy | When embedding unavailable | Effective strategy |
| :--- | :--- | :--- |
| `keyword` | n/a | keyword (FTS) |
| `embedding` | no store/embed service | **falls back to keyword** + warn |
| `hybrid` | no store/embed service | **falls back to keyword** + warn |
| `hybrid` | tcvdb with `nativeHybridSearch` | single server-side hybrid call |
| `hybrid` | sqlite with embedding | keyword + embedding merged with client RRF (`k=60`) |
| invalid string | — | coerced to `"hybrid"` |
Unknown strategy strings are rejected by whitelist and replaced with `"hybrid"`.
---
## `embedding`
Used for **client-side** vectors on the sqlite backend (and L1 dedup). With `storeBackend=tcvdb`, store creation uses `NoopEmbeddingService` and TCVDB server embedding instead.
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `true` (schema) | Forced `false` when `provider` is `"none"` or `"local"`, or remote fields are incomplete |
| `provider` | `string` | `"none"` | See provider table |
| `baseUrl` | `string` | `""` | Required for remote / `qclaw` / `zeroentropy` |
| `apiKey` | `string` | `""` | Required for remote paths |
| `model` | `string` | `""` when none | Required for remote |
| `dimensions` | `number` | `0` when none | Positive int required for remote; `0` defers sqlite-vec table creation |
| `sendDimensions` | `boolean` | `true` | Omit `dimensions` in request body when `false` (BGE-M3 and similar) |
| `proxyUrl` | `string` | omit | **Required** when `provider=qclaw` |
| `conflictRecallTopK` | `number` | `5` | Dedup candidate pool |
| `maxInputChars` | `number` | `5000` | Truncate with warn |
| `timeoutMs` | `number` | `10000` | Per-call timeout (retries up to 3) |
| `recallTimeoutMs` | `number` | omit | Overrides `timeoutMs` on recall path |
| `captureTimeoutMs` | `number` | omit | Overrides `timeoutMs` on capture/dedup path |
### Provider rules
| `provider` | Required fields | Wire behavior |
| :--- | :--- | :--- |
| `"none"` (default) | none | Embedding **disabled**. `dimensions=0` → vec0 tables deferred |
| `"local"` | — | **Not exposed**. Treated as disabled; `configError` set; use a remote provider instead |
| `"qclaw"` | `proxyUrl`, `baseUrl`, `apiKey`, `model`, `dimensions>0` | Request to `proxyUrl` with `Remote-URL: {baseUrl}/embeddings` |
| `"zeroentropy"` | same remote quadruplet | `POST {baseUrl}/models/embed` with `input_type: "query"` |
| any other string (e.g. `openai`, `deepseek`) | `apiKey`, `baseUrl`, `model`, `dimensions>0` | OpenAI-compatible `POST {baseUrl}/embeddings` |
Incomplete remote / `qclaw` config **does not throw**: `embedding.enabled=false`, `embedding.configError` set, plugin continues. Startup logs:
```text
[memory-tdai] [EMBEDDING CONFIG ERROR] Remote embedding provider '…' requires …
```
`sendDimensions: false` example for fixed-dimension models:
```json
{
"embedding": {
"enabled": true,
"provider": "openai",
"baseUrl": "http://localhost:8080/v1",
"apiKey": "",
"model": "bge-m3",
"dimensions": 1024,
"sendDimensions": false
}
}
```
---
## `tcvdb`
Only used when `storeBackend` is `"tcvdb"`.
| Key | Type | Default | Required |
| :--- | :--- | :--- | :--- |
| `url` | `string` | `""` | **Yes** |
| `apiKey` | `string` | `""` | **Yes** |
| `database` | `string` | `""` | **Yes** (must be set explicitly) |
| `username` | `string` | `"root"` | no |
| `alias` | `string` | `""` | optional label for `database.json` |
| `embeddingModel` | `string` | `"bge-large-zh"` | server-side model |
| `timeout` | `number` | `10000` | ms |
| `caPemPath` | `string` | omit | PEM path for HTTPS |
Hard failures at store create:
- missing `url` or `apiKey`
- missing `database`
---
## `bm25`
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `true` | Local sparse encoder (`@tencentdb-agent-memory/tcvdb-text`) |
| `language` | `"zh" \| "en"` | `"zh"` | Any non-`en` string resolves to `"zh"` |
Used for hybrid sparse vectors, especially with the tcvdb backend.
---
## `report`
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `false` | Emit structured METRIC JSON via logger |
| `type` | `string` | `"local"` | Local structured log path |
---
## `llm` (standalone extraction LLM)
When `enabled`, L1/L2/L3 extraction bypasses the host built-in runner (e.g. OpenClaw embedded agent) and calls an OpenAI-compatible API directly.
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `false` | Off → use host LLM |
| `baseUrl` | `string` | `"https://api.openai.com/v1"` | OpenAI-compatible base |
| `apiKey` | `string` | `""` | Required when enabled |
| `model` | `string` | `"gpt-4o"` | Model id |
| `maxTokens` | `number` | `4096` | Max output tokens |
| `timeoutMs` | `number` | `120000` | Request timeout |
| `disableThinking` | `boolean \| string` | `false` | See [disableThinking strategies](#disablethinking-strategies) |
Gateway **top-level** `llm` (in `tdai-gateway.yaml`) is separate: it always configures the standalone runner for Gateway mode and is not gated by `llm.enabled`.
---
## `offload` (Context Offload)
Independent of long-term memory. Default `enabled: false` leaves the context-engine slot and short-term compression off.
| Key | Type | Default | Notes |
| :--- | :--- | :--- | :--- |
| `enabled` | `boolean` | `false` | Master switch |
| `mode` | `"local" \| "backend" \| "collect"` | auto | If omitted: `"backend"` when `backendUrl` set, else `"local"`. **Parser-only** (not always listed in plugin JSON Schema) |
| `model` | `string` | omit | `provider/model`; falls back to host default |
| `temperature` | `number` | `0.2` | LLM temperature |
| `disableThinking` | `boolean \| string` | `false` | Local mode only |
| `forceTriggerThreshold` | `number` | `4` | Force L1 when pending tool pairs ≥ N |
| `dataDir` | `string` | omit | Absolute path; default `~/.openclaw/context-offload` |
| `defaultContextWindow` | `number` | `200000` | Context window size |
| `maxPairsPerBatch` | `number` | `20` | Max tool pairs per L1 batch |
| `l2NullThreshold` | `number` | `4` | Trigger L2 when `node_id=null` count ≥ N |
| `l2TimeoutSeconds` | `number` | `300` | Trigger L2 if last L2 older than N seconds |
| `mildOffloadRatio` | `number` | `0.5` | Mild compression threshold (fraction of window) |
| `aggressiveCompressRatio` | `number` | `0.85` | Aggressive compression threshold |
| `mmdMaxTokenRatio` | `number` | `0.2` | Mermaid injection token budget |
| `backendUrl` | `string` | omit | Routes L1/L1.5/L2/L4 through remote backend |
| `backendApiKey` | `string` | omit | Backend auth token |
| `backendTimeoutMs` | `number` | **`120000` (runtime)** | Plugin schema text may show `10000`; **runtime default is 120000** |
| `offloadRetentionDays` | `number` | `0` | Session/ref/mmd cleanup TTL. `0` = off; `(0,3)` forced to `0`; min effective `3` |
| `logMaxSizeMb` | `number` | `50` | Cap total debug `*.log` size; `0` disables |
| `userId` | `string` | omit | Sent as `X-User-Id` on backend; falls back to primary non-loopback IPv4 |
### Offload modes
| Mode | Behavior |
| :--- | :--- |
| `local` | LLM via AI SDK using `offload.model` or host model |
| `backend` | L1/L1.5/L2/L4 via `backendUrl` |
| `collect` | Async L1/L1.5/L2 data collection; **disables L3 compression** and does **not** occupy the contextEngine slot |
### Offload retention degrade
| Input | Effective |
| :--- | :--- |
| `<= 0` | `0` (disabled) |
| `1` or `2` | `0` (invalid → forced disabled) |
| `>= 3` | as configured |
---
## `disableThinking` strategies
Shared by `llm.disableThinking` and `offload.disableThinking`. Normalization:
| Raw value | Resolved strategy |
| :--- | :--- |
| `false` / omit | `false` (no wrapper) |
| `true` | `"vllm"` (shorthand) |
| known string | that strategy |
| unknown string | `false` + console warn |
| Strategy | Injection on chat-completion body |
| :--- | :--- |
| `vllm` | `chat_template_kwargs.enable_thinking = false` |
| `deepseek` | top-level `enable_thinking: false` |
| `dashscope` | top-level `enable_thinking: false` |
| `openai` | `reasoning_effort: "low"` (cannot fully disable) |
| `anthropic` | `thinking: { type: "disabled" }` |
| `kimi` | same as anthropic |
| `gemini` | `thinking_config: { thinking_budget: 0 }` |
Only requests with a `messages` array are modified; embedding and other non-chat calls pass through unchanged.
---
## Gateway-only config (standalone / Hermes)
`loadGatewayConfig()` layers file + env. Memory fields reuse `parseConfig` under the `memory` key.
| Section | Keys | Env overrides |
| :--- | :--- | :--- |
| `server` | `port` (8420), `host` (`127.0.0.1`), `apiKey?`, `corsOrigins[]` | `TDAI_GATEWAY_PORT`, `TDAI_GATEWAY_HOST`, `TDAI_GATEWAY_API_KEY`, `TDAI_CORS_ORIGINS` |
| `data` | `baseDir` | `TDAI_DATA_DIR`; root via `MEMORY_TENCENTDB_ROOT` (default `~/.memory-tencentdb/memory-tdai`) |
| `llm` | `baseUrl`, `apiKey`, `model`, `maxTokens`, `timeoutMs`, `disableThinking` | `TDAI_LLM_*` |
| `memory` | full plugin schema | none (file only) |
Config file resolution order:
1. `TDAI_GATEWAY_CONFIG`
2. `./tdai-gateway.yaml` or `.json` in CWD
3. `/tdai-gateway.yaml` or `.json`
4. env-only defaults
String leaves of the form `${VAR}` expand from the environment. Auth: when `server.apiKey` is set, all routes except `GET /health` and CORS `OPTIONS` require `Authorization: Bearer …`. Empty `corsOrigins` (default) sends no CORS headers.
---
## Schema vs runtime notes
| Topic | Behavior |
| :--- | :--- |
| Zero-config | `{}` or plugin enabled only is valid |
| Schema `additionalProperties` | `true` on root object |
| Incomplete embedding | Soft-disable + `configError` log; no crash |
| Incomplete tcvdb | Hard error at store create → degraded store |
| Aggressive L0/L1 cleanup | Requires explicit `allowAggressiveCleanup` |
| Offload vs memory | Independent switches; offload default off |
| `provider=local` embedding | Rejected at config entry; internal local code not reachable from user config |
| `backendTimeoutMs` | Trust **runtime** default `120000` over schema text |
| Parser-only offload keys | `mode`, `offloadRetentionDays`, `logMaxSizeMb`, `userId` exist in `parseConfig` even if the OpenClaw schema omits some |
---
## Example configurations
```json
{
"memory-tencentdb": {
"enabled": true
}
}
```
```json
{
"memory-tencentdb": {
"timezone": "Asia/Shanghai",
"storeBackend": "sqlite",
"capture": {
"enabled": true,
"excludeAgents": ["bench-judge-*"],
"l0l1RetentionDays": 90,
"cleanTime": "03:00"
},
"extraction": {
"enabled": true,
"enableDedup": true,
"maxMemoriesPerSession": 20
},
"pipeline": {
"everyNConversations": 5,
"enableWarmup": true,
"l1IdleTimeoutSeconds": 600
},
"recall": {
"enabled": true,
"maxResults": 5,
"scoreThreshold": 0.3,
"strategy": "hybrid",
"timeoutMs": 5000
},
"embedding": {
"enabled": true,
"provider": "openai",
"baseUrl": "https://api.openai.com/v1",
"apiKey": "${EMBEDDING_API_KEY}",
"model": "text-embedding-3-small",
"dimensions": 1536,
"sendDimensions": true
},
"bm25": { "enabled": true, "language": "zh" },
"llm": { "enabled": false },
"offload": { "enabled": false }
}
}
```
```json
{
"memory-tencentdb": {
"storeBackend": "tcvdb",
"tcvdb": {
"url": "http://10.0.1.1:8100",
"username": "root",
"apiKey": "",
"database": "agent_memory_prod",
"embeddingModel": "bge-large-zh",
"timeout": 10000
},
"bm25": { "enabled": true, "language": "zh" },
"recall": { "strategy": "hybrid" }
}
}
```
```json
{
"memory-tencentdb": {
"llm": {
"enabled": true,
"baseUrl": "https://api.openai.com/v1",
"apiKey": "",
"model": "gpt-4o-mini",
"maxTokens": 4096,
"timeoutMs": 120000,
"disableThinking": false
},
"offload": {
"enabled": true,
"mode": "local",
"temperature": 0.2,
"forceTriggerThreshold": 4,
"mildOffloadRatio": 0.5,
"aggressiveCompressRatio": 0.85
}
}
}
```
Treat `apiKey` values as secrets. Prefer environment injection; do not paste real keys into chat logs or diagnostics packages without redaction.
---
## Verification signals
After config changes, restart the host Gateway and check:
| Signal | Healthy reading |
| :--- | :--- |
| Log prefix | `[memory-tdai]` config dump includes capture/recall/extraction/pipeline/offload |
| Embedding error | No `[EMBEDDING CONFIG ERROR]` unless intentional keyword-only mode |
| Store log | `Store initialized: backend=sqlite|tcvdb, provider=…` |
| Data dir (OpenClaw) | `~/.openclaw/.../memory-tdai/` with `conversations/`, `records/`, `scene_blocks/`, `vectors.db` (sqlite) |
| Offload | Mermaid injection + files under `~/.openclaw/context-offload` only when `offload.enabled` |
| Tools | `tdai_memory_search` / `tdai_conversation_search` return results |
---
## Related pages
sqlite vs tcvdb, embedding quadruplets, BM25 language, store health checks.
Turn on offload.enabled, contextEngine slot, after-tool-call patch.
Zero-config enable, gateway restart, smoke checks.
Standalone Hermes sidecar endpoints and auth.
Silent plugin, no recall, embedding degrade, aggressive cleanup.
On-disk layout for memory-tdai and context-offload trees.
---
## 14. Agent tools
> Registered tools tdai_memory_search and tdai_conversation_search: parameters, hybrid/embedding/keyword strategies, RRF merge, and combined 3-calls-per-turn limit.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/14-agent-tools.md
- Generated: 2026-07-09T07:34:52.351Z
### Source Files
- `index.ts`
- `src/core/tools/memory-search.ts`
- `src/core/tools/conversation-search.ts`
- `openclaw.plugin.json`
- `src/core/store/search-utils.ts`
---
title: "Agent tools"
description: "Registered tools tdai_memory_search and tdai_conversation_search: parameters, hybrid/embedding/keyword strategies, RRF merge, and combined 3-calls-per-turn limit."
---
OpenClaw registers two agent-callable tools — `tdai_memory_search` (L1 structured memories) and `tdai_conversation_search` (L0 raw messages) — via `api.registerTool()` in the plugin entrypoint. Both share the same host-neutral search path: `TdaiCore.searchMemories` / `searchConversations` → `executeMemorySearch` / `executeConversationSearch`, with automatic hybrid FTS + embedding retrieval and Reciprocal Rank Fusion (RRF). Tool contracts are declared in `openclaw.plugin.json` as `contracts.tools`.
## When tools register
| Condition | Behavior |
| --- | --- |
| `recall.enabled \|\| capture.enabled` | Both tools register |
| Both `recall.enabled` and `capture.enabled` are false | Tools are **not** registered; debug log: `Memory tools … not registered — memory features disabled` |
Registration is host-side OpenClaw only. Hermes exposes parallel tools under different names (see [Host surfaces](#host-surfaces)).
## Tool inventory
| Tool | Layer | Purpose |
| --- | --- | --- |
| `tdai_memory_search` | L1 | Long-term structured memories (preferences, events, instructions) |
| `tdai_conversation_search` | L0 | Raw dialogue messages (exact wording, timeline, session context) |
Recommended order for the model: try `tdai_memory_search` first; fall back to `tdai_conversation_search` when structured memories are insufficient or exact quotes are needed. Auto-recall also injects a `` block that documents this order and the per-turn call budget.
```mermaid
flowchart LR
subgraph agent["Agent turn"]
T1["tdai_memory_search"]
T2["tdai_conversation_search"]
end
subgraph core["TdaiCore"]
SM["searchMemories"]
SC["searchConversations"]
end
subgraph exec["Tool executors"]
EM["executeMemorySearch"]
EC["executeConversationSearch"]
end
subgraph store["IMemoryStore + EmbeddingService"]
FTS["searchL0Fts / searchL1Fts"]
VEC["searchL0Vector / searchL1Vector"]
end
T1 --> SM --> EM
T2 --> SC --> EC
EM --> FTS
EM --> VEC
EC --> FTS
EC --> VEC
```
## `tdai_memory_search`
Searches L1 memory records. OpenClaw label: **Memory Search**.
### Parameters
| Name | Type | Required | Default / bounds | Description |
| --- | --- | --- | --- | --- |
| `query` | `string` | yes | — | What to recall about the user |
| `limit` | `number` | no | `5`, clamped to `[1, 20]` | Max results returned |
| `type` | `string` enum | no | — | `persona` \| `episodic` \| `instruction` (exact match after merge) |
| `scene` | `string` | no | — | Case-insensitive substring match on `scene_name` |
### Response shape (to the agent)
Text content is produced by `formatSearchResponse`:
- Configuration failure: plain message asking for embedding / FTS setup
- Empty: `No matching memories found.`
- Hits: `Found N matching memories:` plus lines shaped as
`- **[type]** (priority: …) [scene: …] (score: 0.xxx)` and content
OpenClaw tool `details`: `{ count, strategy }` on success; `{ error }` on failure.
### Result item fields
| Field | Meaning |
| --- | --- |
| `id` | L1 `record_id` |
| `content` | Memory text |
| `type` | `persona` / `episodic` / `instruction` |
| `priority` | Priority; negative values format as `(global instruction)` |
| `scene_name` | Scene association |
| `score` | Rank score (RRF score when hybrid) |
| `created_at` / `updated_at` | From store timestamps (`timestamp_start` / `timestamp_end`) |
## `tdai_conversation_search`
Searches L0 conversation messages. OpenClaw label: **Conversation Search**.
### Parameters
| Name | Type | Required | Default / bounds | Description |
| --- | --- | --- | --- | --- |
| `query` | `string` | yes | — | Conversation content to find |
| `limit` | `number` | no | `5`, clamped to `[1, 20]` | Max messages returned |
| `session_key` | `string` | no | — | Exact filter on `session_key` (applied after merge) |
### Response shape (to the agent)
From `formatConversationSearchResponse`:
- Configuration failure: plain embedding/FTS setup message
- Empty: `No matching conversation messages found.`
- Hits: `Found N matching message(s):` with blocks
`**[role]** Session: … [recorded_at] (score: …)` and full message text
OpenClaw tool `details`: `{ count }` on success (strategy is not currently returned in `details`, though the executor logs it).
### Result item fields
| Field | Meaning |
| --- | --- |
| `id` | L0 `record_id` |
| `session_key` | Session identity |
| `role` | `user` or `assistant` |
| `content` | Single-message text (`message_text`) |
| `score` | Rank score (RRF when hybrid) |
| `recorded_at` | Capture timestamp |
## Search strategies
Agent tools do **not** take a `strategy` parameter. Capability and result presence choose the effective strategy:
| Effective `strategy` | When |
| --- | --- |
| `hybrid` | Both FTS and vector paths return at least one candidate |
| `embedding` | Only vector path returns candidates (or both empty while embedding is available) |
| `fts` | Only FTS path returns candidates (or both empty while only FTS is available) |
| `none` | Empty query, missing store, or neither embedding nor FTS available |
### Execution pipeline
1. Validate `query` (non-empty after trim) and `vectorStore`.
2. Probe capabilities: `embeddingService` present → vector path; `vectorStore.isFtsAvailable()` → FTS path.
3. Over-retrieve candidates:
- L1: `candidateK = limit * 3`
- L0: `candidateK = limit * 3`, or `limit * 4` when `session_key` filter is set
4. Run FTS and vector searches in parallel (`Promise.all`).
5. Merge with RRF when both lists are non-empty; otherwise keep the single non-empty list.
6. Apply secondary filters (`type` / `scene` for L1; `session_key` for L0).
7. Slice to `limit`.
### FTS (keyword) path
- Query normalization: `buildFtsQuery(query)` in the SQLite store layer.
- Tokenization: jieba `cutForSearch` when available (Chinese-friendly, stop-word filtered); otherwise Unicode word split.
- Output form: quoted tokens joined with `OR` (e.g. `"北京" OR "烤鸭"`). Empty tokenization → skip FTS for that call.
- Store APIs: `searchL1Fts` / `searchL0Fts`.
### Embedding path
- Embed the raw `query` via `EmbeddingService.embed`.
- Store APIs: `searchL1Vector` / `searchL0Vector` with the embedding and original query string.
- Failures are non-fatal: that path returns `[]` and the other path may still succeed.
### Failure / degrade messages
When neither embedding nor FTS is available, tools return `strategy: "none"` and a message that memory/conversation search requires an embedding provider or FTS5 support (configure `embedding.provider`, e.g. `openai_compatible`).
Empty query or missing store returns empty results with `strategy: "none"` without that configuration message.
## RRF merge
Both tools use Reciprocal Rank Fusion with constant **`RRF_K = 60`**:
```text
score(item) = Σ over lists 1 / (K + rank + 1) // rank is 0-based
```
- Items are keyed by `id` (`record_id`).
- Scores sum across lists when an item appears in both FTS and vector rankings.
- Output is sorted by descending RRF score; each item’s `score` field is replaced with the RRF score for consistent ranking semantics.
- Shared helper `rrfMerge` / `RRF_K` also lives in `src/core/store/search-utils.ts` for other hybrid call sites (tool modules currently inline equivalent L0/L1 helpers).
## Combined 3-calls-per-turn limit
| Aspect | Behavior |
| --- | --- |
| Scope | `tdai_memory_search` **and** `tdai_conversation_search` **combined** |
| Budget | **3** calls per agent turn |
| Enforcement today | **Soft** — model-facing only |
| Hard gate | **Not implemented** — entrypoint TODO notes a future `before_tool_call` + early-return hard limit |
How the soft limit is communicated:
1. **Tool descriptions** on both OpenClaw tools:
`Limit: tdai_memory_search and tdai_conversation_search share a combined limit of 3 calls per turn. Stop searching after 3 total attempts.`
2. **Auto-recall guide** (`MEMORY_TOOLS_GUIDE` in `auto-recall`): injected as system context when recall runs; tells the model to retry with different keywords/tools within the budget, then stop and answer from existing context.
There is no runtime counter rejection in the tool `execute` handlers yet. Hosts that need hard throttling must implement it outside these executors until the planned hook lands.
## Host surfaces
### OpenClaw plugin
| Item | Value |
| --- | --- |
| Tool names | `tdai_memory_search`, `tdai_conversation_search` |
| Contract | `openclaw.plugin.json` → `contracts.tools` |
| Metrics | `report("tool_call", { tool, query, limit, …, success, durationMs, … })` |
| Errors | Text: `Memory search failed: …` / `Conversation search failed: …` |
### Gateway HTTP (Hermes sidecar / standalone)
Same core methods, different wire format:
| Endpoint | Maps to |
| --- | --- |
| `POST /search/memories` | `core.searchMemories` — body: `query`, optional `limit`, `type`, `scene` |
| `POST /search/conversations` | `core.searchConversations` — body: `query`, optional `limit`, `session_key` |
Responses: `{ results: string, total, strategy? }` where `results` is the **formatted text** (not a structured array). Missing `query` → HTTP 400.
### Hermes MemoryProvider
| Hermes tool name | OpenClaw equivalent | Notes |
| --- | --- | --- |
| `memory_tencentdb_memory_search` | `tdai_memory_search` | Args: `query`, `limit`, `type` (no `scene` in Hermes schema) |
| `memory_tencentdb_conversation_search` | `tdai_conversation_search` | Args: `query`, `limit` (no `session_key` in Hermes schema) |
`limit` is coerced to `[1, 20]` (default 5) with defensive parsing for string/float LLM args. Old `tdai_*` names are **not** registered on Hermes; unknown names return an error JSON. Tools require a connected Gateway (circuit breaker and reconnect probes apply).
## Verification signals
| Signal | Meaning |
| --- | --- |
| Log `[memory-tdai] [tool] tdai_memory_search called:` | OpenClaw tool invoked |
| Log `strategy=hybrid\|embedding\|fts` | Effective retrieval path |
| Log `[hybrid] RRF merged: fts=…, vec=…` | Both paths contributed |
| Tool text `Found N matching memories:` / `message(s):` | Successful hit formatting |
| Message about embedding / FTS not configured | Degraded setup |
| Debug: tools not registered — memory features disabled | Both `recall` and `capture` off |
Smoke from an agent turn (OpenClaw): call `tdai_memory_search` with a known seeded fact, then `tdai_conversation_search` for the source wording. Stay within three total attempts per turn.
## Related pages
L0 conversation vs L1 atom records that these tools search.
sqlite-vec + FTS5 vs tcvdb, and embedding provider setup required for hybrid search.
`POST /search/memories` and `POST /search/conversations` request/response fields.
`recall`, `capture`, `embedding`, and store settings that gate tool registration and capability.
First smoke checks with tdai tools after enabling the plugin.
No recall, embedding degrade, and recovery probes when tools return empty or setup messages.
---
## 15. Gateway HTTP API
> Standalone Hermes sidecar endpoints: GET /health, POST /recall, /capture, /search/memories, /search/conversations, /session/end, /seed — request/response fields and auth.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/15-gateway-http-api.md
- Generated: 2026-07-09T07:34:58.178Z
### Source Files
- `src/gateway/server.ts`
- `src/gateway/types.ts`
- `src/gateway/config.ts`
- `hermes-plugin/memory/memory_tencentdb/client.py`
- `src/adapters/standalone/host-adapter.ts`
---
title: "Gateway HTTP API"
description: "Standalone Hermes sidecar endpoints: GET /health, POST /recall, /capture, /search/memories, /search/conversations, /session/end, /seed — request/response fields and auth."
---
The **TDAI Gateway** (`TdaiGateway` in `src/gateway/server.ts`) is a Node.js native `http` sidecar that exposes TDAI Core over JSON HTTP for standalone and Hermes hosts. Default bind is `http://127.0.0.1:8420`. It does not use Express or Fastify. Hermes talks to it through `MemoryTencentdbSdkClient` (`hermes-plugin/memory/memory_tencentdb/client.py`); OpenClaw uses the in-process plugin path and does not require this API.
```mermaid
flowchart LR
subgraph Host["Host process"]
H["Hermes MemoryTencentdbProvider"]
C["MemoryTencentdbSdkClient"]
S["GatewaySupervisor"]
H --> C
H --> S
end
subgraph GW["TdaiGateway :8420"]
R["Route handlers"]
A["StandaloneHostAdapter"]
Core["TdaiCore"]
R --> Core
Core --> A
end
subgraph Store["Data dir"]
D["~/.memory-tencentdb/memory-tdai\nor TDAI_DATA_DIR"]
end
C -->|"HTTP JSON\nBearer optional"| R
S -->|"spawn / health"| GW
Core --> D
```
## Endpoint inventory
| Method | Path | Auth when `apiKey` set | Purpose |
| :--- | :--- | :--- | :--- |
| `GET` | `/health` | Never | Liveness / store readiness |
| `OPTIONS` | `*` | Never | CORS preflight → `204` |
| `POST` | `/recall` | Yes | Prefetch memory context for a turn |
| `POST` | `/capture` | Yes | Record a user/assistant turn (L0 + pipeline notify) |
| `POST` | `/search/memories` | Yes | Search L1 structured memories |
| `POST` | `/search/conversations` | Yes | Search L0 raw conversations |
| `POST` | `/session/end` | Yes | End session and flush pipeline work |
| `POST` | `/seed` | Yes | Batch seed historical conversations (blocking) |
All successful responses use `Content-Type: application/json`. Unknown routes return `404` with `{ "error": "Not found: METHOD /path" }`. Uncaught handler errors return `500` with `{ "error": "" }`. Invalid JSON body fails with `500` and message `Invalid JSON body` (parse rejection is not mapped to `400`).
## Runtime and config surface
| Concern | Default | Config / env |
| :--- | :--- | :--- |
| Port | `8420` | `server.port` / `TDAI_GATEWAY_PORT` |
| Host | `127.0.0.1` | `server.host` / `TDAI_GATEWAY_HOST` |
| API key | unset (auth off) | `server.apiKey` / `TDAI_GATEWAY_API_KEY` |
| CORS allow-list | `[]` (no CORS headers) | `server.corsOrigins` / `TDAI_CORS_ORIGINS` |
| Data dir | `$MEMORY_TENCENTDB_ROOT/memory-tdai` or `~/.memory-tencentdb/memory-tdai` (legacy `~/memory-tdai` if present) | `data.baseDir` / `TDAI_DATA_DIR` |
| LLM | OpenAI-compatible defaults | `llm.*` / `TDAI_LLM_*` |
| Memory plugin config | Same schema as OpenClaw plugin | `memory` block via `parseConfig` |
Config file resolution order:
1. `TDAI_GATEWAY_CONFIG` (explicit path)
2. `./tdai-gateway.yaml` or `./tdai-gateway.json` in CWD
3. `/tdai-gateway.yaml` or `.json`
4. Environment-only config
String leaves of the form `${VAR}` expand from the process environment. Malformed config files fall back to env-only defaults.
Start (manual):
```bash
npx tsx src/gateway/server.ts
# or via Hermes supervisor / MEMORY_TENCENTDB_GATEWAY_CMD
```
Hermes default client base URL: `http://127.0.0.1:8420`. Client default timeout: **10s** (`DEFAULT_TIMEOUT`); health probes often use **2–3s**; seed uses **300s**.
## Authentication
Auth is **opt-in**. When `server.apiKey` / `TDAI_GATEWAY_API_KEY` is unset, every route is open (legacy default). Startup logs:
- `Security posture: auth=disabled|ENABLED (Bearer) host=... cors=...`
- `WARN` if auth is off
- Louder `WARN` if bound to a non-loopback host without an API key
- `WARN` if CORS allow-list contains `*`
When an API key **is** set:
- `GET /health` and `OPTIONS` stay open
- All other routes require `Authorization: Bearer `
- Missing/malformed header → `401` `{ "error": "Unauthorized: missing Bearer token" }`
- Wrong token → `401` `{ "error": "Unauthorized: invalid token" }`
- Comparison uses constant-time equality (`crypto.timingSafeEqual`) after length check
### Client-side keys (Hermes)
| Process | Variable | Role |
| :--- | :--- | :--- |
| Gateway (server) | `TDAI_GATEWAY_API_KEY` or `server.apiKey` | Enforces Bearer check |
| Hermes client | `MEMORY_TENCENTDB_GATEWAY_API_KEY`, fallback `TDAI_GATEWAY_API_KEY` | Attaches Bearer on outbound calls |
The supervisor does **not** inject the client key into the Gateway process. Operators must set the same secret on both sides when auth is enabled.
```bash
curl -s http://127.0.0.1:8420/health
curl -s -X POST http://127.0.0.1:8420/recall \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TDAI_GATEWAY_API_KEY" \
-d '{"query":"...","session_key":"..."}'
```
## CORS
| `corsOrigins` | Behavior |
| :--- | :--- |
| `[]` (default) | No `Access-Control-Allow-*` headers; browsers block cross-origin use |
| concrete origins | Echo matching `Origin`; set methods `GET, POST, OPTIONS`, headers `Content-Type, Authorization`, and `Vary: Origin` |
| `["*"]` | `Access-Control-Allow-Origin: *` (dev only; startup WARN) |
`OPTIONS` always answers `204` after CORS header application (no auth gate).
## Hermes lifecycle mapping
| Hermes / provider call | Gateway endpoint | Notes |
| :--- | :--- | :--- |
| `prefetch(query)` | `POST /recall` | Synchronous; inject returned `context` |
| `sync_turn(user, assistant)` | `POST /capture` | Often fire-and-forget on a background thread |
| LLM tools (memory / conversation search) | `POST /search/memories`, `POST /search/conversations` | Via `MemoryTencentdbSdkClient` |
| `on_session_end` / shutdown flush | `POST /session/end` | Flush pending pipeline work |
| Batch import (optional) | `POST /seed` | Not the primary turn path; long-running |
Provider-side reliability (circuit breaker, capture back-pressure, supervisor) lives in the Hermes plugin, not in the HTTP handlers themselves.
---
## Endpoints
:::endpoint GET /health Liveness and store readiness
Always public. Used by `GatewaySupervisor`, Docker/k8s probes, and `memory-tencentdb-ctl health`.
**Response `200`**
| Field | Type | Description |
| :--- | :--- | :--- |
| `status` | `"ok"` \| `"degraded"` | `"ok"` when vector store is present; otherwise `"degraded"` |
| `version` | string | Gateway version constant (`0.1.0` in current source) |
| `uptime` | number | Seconds since listen |
| `stores.vectorStore` | boolean | `TdaiCore.getVectorStore()` truthy |
| `stores.embeddingService` | boolean | `TdaiCore.getEmbeddingService()` truthy |
```json
{
"status": "ok",
"version": "0.1.0",
"uptime": 42,
"stores": {
"vectorStore": true,
"embeddingService": true
}
}
```
:::
:::endpoint POST /recall Prefetch memory context for the next turn
Calls `TdaiCore.handleBeforeRecall(query, session_key)`.
Recall query text (usually the latest user message).
Session scope key.
Optional; accepted by types and Hermes client. Current handler does not pass it into core.
**Errors:** `400` if `query` or `session_key` missing.
**Response `200`**
| Field | Type | Description |
| :--- | :--- | :--- |
| `context` | string | System context to inject (`appendSystemContext`, may be empty) |
| `strategy` | string? | Recall strategy used |
| `memory_count` | number? | Count of recalled L1 memories |
```bash
curl -s -X POST http://127.0.0.1:8420/recall \
-H "Content-Type: application/json" \
-d '{"query":"What did we decide about storage?","session_key":"sess-1"}'
```
```json
{
"context": "...memory context...",
"strategy": "hybrid",
"memory_count": 3
}
```
:::
:::endpoint POST /capture Record a conversation turn
Calls `TdaiCore.handleTurnCommitted`. If `messages` is omitted, the handler synthesizes a two-message array from `user_content` / `assistant_content`.
User turn text.
Assistant turn text.
Session scope key.
Optional session id passed through to core.
Optional; accepted by types/client; not applied by the current handler.
Optional full message list; defaults to user + assistant pair built from content fields.
**Errors:** `400` if any of `user_content`, `assistant_content`, `session_key` missing.
**Response `200`**
| Field | Type | Description |
| :--- | :--- | :--- |
| `l0_recorded` | number | L0 records written this turn |
| `scheduler_notified` | boolean | Whether the extraction pipeline scheduler was notified |
```bash
curl -s -X POST http://127.0.0.1:8420/capture \
-H "Content-Type: application/json" \
-d '{
"user_content": "Remember I prefer SQLite for local dev",
"assistant_content": "Noted — local SQLite, TCVDB for prod.",
"session_key": "sess-1"
}'
```
```json
{
"l0_recorded": 1,
"scheduler_notified": true
}
```
:::
:::endpoint POST /search/memories Search L1 structured memories
Calls `TdaiCore.searchMemories`. Hermes client default `limit` is `5`.
Search query.
Max results.
Optional L1 type filter.
Optional scene filter.
**Errors:** `400` if `query` missing.
**Response `200`**
| Field | Type | Description |
| :--- | :--- | :--- |
| `results` | string | Formatted result text from core |
| `total` | number | Hit count |
| `strategy` | string | Search strategy used |
:::
:::endpoint POST /search/conversations Search L0 raw conversations
Calls `TdaiCore.searchConversations`.
Search query.
Max results.
Optional session scope for the search.
**Errors:** `400` if `query` missing.
**Response `200`**
| Field | Type | Description |
| :--- | :--- | :--- |
| `results` | string | Formatted result text |
| `total` | number | Hit count |
:::
:::endpoint POST /session/end End session and flush
Calls `TdaiCore.handleSessionEnd(session_key)`. Response always reports success once the call returns.
Session to flush/end.
Optional; accepted by types/client; not applied by the current handler.
**Errors:** `400` if `session_key` missing.
**Response `200`**
| Field | Type | Description |
| :--- | :--- | :--- |
| `flushed` | boolean | Always `true` on successful completion |
:::
:::endpoint POST /seed Batch seed historical conversations
Validates input with the same seed pipeline as CLI seed (`validateAndNormalizeRaw` / `executeSeed`). **Blocking** — large inputs may take minutes. Hermes client default timeout is **300s**.
Output is written under a timestamped directory:
`{data.baseDir}/seed-YYYYMMDD-HHMMSS`
Gateway injects its LLM settings into the seed `pluginConfig` (`llm.enabled=true` plus baseUrl/apiKey/model/maxTokens/timeoutMs/disableThinking). Optional `config_override` is deep-merged one level per top-level key.
Seed payload — Format A `{ "sessions": [...] }` or Format B `[...]` (same shapes as `openclaw memory-tdai seed --input`).
Fallback session key when input sessions omit one.
Require each round to include both user and assistant (default false).
Auto-fill missing timestamps (default **true**).
Plugin config overrides deep-merged onto gateway memory + injected LLM config.
**Errors:**
- `400` `{ "error": "Missing required field: data" }`
- `400` validation failure: `{ "error": "", "validation_errors": [...] }` (`SeedValidationError`)
**Response `200`**
| Field | Type | Description |
| :--- | :--- | :--- |
| `sessions_processed` | number | Sessions completed |
| `rounds_processed` | number | Rounds completed |
| `messages_processed` | number | Messages completed |
| `l0_recorded` | number | L0 records written |
| `duration_ms` | number | Wall time |
| `output_dir` | string | Seed output directory path |
```bash
curl -s -X POST http://127.0.0.1:8420/seed \
-H "Content-Type: application/json" \
-d '{
"data": {
"sessions": [{
"sessionKey": "import-1",
"conversations": [[
{"role":"user","content":"Hello"},
{"role":"assistant","content":"Hi"}
]]
}]
},
"auto_fill_timestamps": true
}'
```
:::
## Error envelope
Standard error body (`GatewayErrorResponse`):
```json
{ "error": "string", "code": "optional string" }
```
`code` is defined on the type but handlers typically set only `error`. Seed validation adds `validation_errors` outside that minimal shape.
| Status | When |
| :--- | :--- |
| `400` | Missing required fields; seed validation failure |
| `401` | Auth enabled and Bearer missing/invalid |
| `404` | Unknown method/path |
| `500` | Uncaught errors, including invalid JSON body |
## Host adapter boundary
`TdaiGateway` constructs `StandaloneHostAdapter` with:
- `dataDir` → gateway `data.baseDir`
- `llmConfig` → gateway `llm`
- `platform: "gateway"`
- default `userId`: `"default_user"`
Per-request handlers currently scope work primarily via `session_key` (and capture `session_id` / message payload). This is the standalone/Hermes path; OpenClaw uses a different host adapter and never needs these HTTP routes for normal plugin operation.
## Smoke checks
```bash
# Health (always works without Bearer)
curl -s http://127.0.0.1:8420/health | python3 -m json.tool
# Recall with auth when configured
curl -s -X POST http://127.0.0.1:8420/recall \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TDAI_GATEWAY_API_KEY" \
-d '{"query":"test","session_key":"debug"}'
```
Expect health `status` of `"ok"` or `"degraded"`. On Hermes, supervisor treats either as “running”. Startup logs with tag `[tdai-gateway]` should show listen address and security posture.
## Failure modes (API-relevant)
| Symptom | Likely cause | Probe |
| :--- | :--- | :--- |
| Connection refused | Gateway not started / wrong port | Supervisor, `MEMORY_TENCENTDB_GATEWAY_CMD`, port env |
| `401` on POST | Client missing Bearer while Gateway has `apiKey` | Align `TDAI_GATEWAY_API_KEY` and `MEMORY_TENCENTDB_GATEWAY_API_KEY` |
| Health `"degraded"` | Vector store not ready | `stores` object in `/health`; storage config |
| Hermes pauses calls | Provider circuit breaker (plugin-side) | Hermes provider logs; not an HTTP status from Gateway |
| Seed times out | Large batch + default 300s client timeout | Increase client timeout; run offline seed CLI instead |
| Non-loopback + no auth | Misconfiguration exposure | Startup WARN; bind `127.0.0.1` or set API key |
## Related pages
Provider install, Gateway supervision, and config.yaml keys.
memory-tencentdb-ctl start/stop/status/health/logs and path overrides.
StandaloneHostAdapter vs OpenClaw adapter boundary.
Format A/B seed input and L0→L3 verification (CLI and /seed parity).
Full memory.* schema used under gateway memory config.
Gateway circuit breaker and recovery probes from the host side.
---
## 16. CLI reference
> openclaw memory-tdai seed flags, and package bins migrate-sqlite-to-tcvdb, export-tencent-vdb, read-local-memory including build prerequisites.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/16-cli-reference.md
- Generated: 2026-07-09T07:35:47.626Z
### Source Files
- `src/cli/index.ts`
- `src/cli/commands/seed.ts`
- `src/cli/README.md`
- `package.json`
- `bin/migrate-sqlite-to-tcvdb.mjs`
- `bin/export-tencent-vdb.mjs`
- `bin/read-local-memory.mjs`
---
title: "CLI reference"
description: "openclaw memory-tdai seed flags, and package bins migrate-sqlite-to-tcvdb, export-tencent-vdb, read-local-memory including build prerequisites."
---
The package `@tencentdb-agent-memory/memory-tencentdb` exposes two CLI surfaces: the OpenClaw plugin command namespace `openclaw memory-tdai` (currently `seed` only), and three npm `bin` entry points—`migrate-sqlite-to-tcvdb`, `export-tencent-vdb`, and `read-local-memory`—each a thin Node launcher over a TypeScript build product under `scripts/*/dist/`.
## CLI inventory
| Surface | How to invoke | Build / host requirement |
|---|---|---|
| `openclaw memory-tdai seed` | OpenClaw plugin CLI (`api.registerCli`) | Plugin installed/loaded; no separate script build |
| `migrate-sqlite-to-tcvdb` | Package bin or `npm run migrate-sqlite-to-tcvdb -- …` | `npm run build:migrate-sqlite-to-vdb` |
| `export-tencent-vdb` | Package bin or `npm run export-tencent-vdb -- …` | `npm run build:export-tencent-vdb` |
| `read-local-memory` | Package bin or `npm run read-local-memory -- …` | `npm run build:read-local-memory` |
```text
OpenClaw host Package bins (node)
───────────────── ───────────────────
openclaw memory-tdai seed migrate-sqlite-to-tcvdb
→ registerMemoryTdaiCli export-tencent-vdb
→ registerSeedCommand read-local-memory
→ executeSeed (L0→L1, L2/L3 wired) → scripts/*/dist/*.js
```
`memory-tencentdb-ctl` and `export-diagnostic.sh` are shell ops tools under `scripts/`, not npm `bin` entries. See [Gateway control](/gateway-control) and [Export diagnostics](/export-diagnostics).
## Build prerequisites
| Requirement | Value |
|---|---|
| Node | `>=22.16.0` (`package.json` `engines`) |
| TypeScript toolchain | Dev deps: `typescript`, `tsdown`, `@types/node` (for local builds) |
| Full scripts build | `npm run build:scripts` (runs all three script builds) |
| Full package build | `npm run build` → `build:plugin` + `build:scripts` |
| Publish path | `prepack` runs `npm run build`; published `files` include `bin/` and `scripts/*/dist/` |
### Per-bin compile targets
| Bin | npm script | Dist entry loaded by bin |
|---|---|---|
| `migrate-sqlite-to-tcvdb` | `build:migrate-sqlite-to-vdb` | `scripts/migrate-sqlite-to-tcvdb/dist/scripts/migrate-sqlite-to-tcvdb/cli-entry.js` |
| `export-tencent-vdb` | `build:export-tencent-vdb` | `scripts/export-tencent-vdb/dist/export-tencent-vdb.js` |
| `read-local-memory` | `build:read-local-memory` | `scripts/read-local-memory/dist/read-local-memory.js` |
```bash
# From package root (source checkout)
npm install
npm run build:scripts
# Or individually
npm run build:migrate-sqlite-to-vdb
npm run build:export-tencent-vdb
npm run build:read-local-memory
```
`export-tencent-vdb` and `read-local-memory` bins exit with code `1` if the dist file is missing and print the required `npm run build:…` command. `migrate-sqlite-to-tcvdb` does not preflight the dist path; a missing build fails on `import()`.
After a normal `npm install @tencentdb-agent-memory/memory-tencentdb`, bins resolve from the package and use published dist artifacts (no local `tsc` required).
---
## `openclaw memory-tdai seed`
Registers under the plugin CLI namespace `memory-tdai` via `api.registerCli` (both full runtime and `registrationMode === "cli-metadata"` discovery). Implementation: `src/cli/index.ts` → `registerSeedCommand` → `executeSeed`.
```bash
openclaw memory-tdai seed --input [options]
```
### Flags
Path to input JSON file (Format A or Format B).
Pipeline data directory. Default: `/memory-tdai-seed-` where `stateDir` is the OpenClaw state root (for example `~/.openclaw`).
Fallback session key when an input session omits `sessionKey`.
JSON config override path. Two-level deep-merged on top of the live plugin config from `openclaw.json`.
Require every round to include both `user` and `assistant` messages.
Skip interactive confirmations (timestamp auto-fill).
### Behavior notes
| Topic | Behavior |
|---|---|
| Pipeline | Feeds L0 capture then L1 extraction; L2/L3 runners are wired, but seed currently waits only for L1 idle before destroy—L2/L3 artifacts may still be incomplete |
| Timestamps | All messages must have timestamps or none must; if all missing, prompts to fill with current time (or auto-fills with `--yes`) |
| Mixed timestamps | Fatal validation error |
| Output dir exists + non-empty | Fatal unless empty; checkpoint resume is **not** implemented (exit if `.metadata/checkpoint.json` exists) |
| Config merge | Object keys merge one level deep; non-objects are replaced |
| Progress | Stdout progress line per round; final box summary: sessions, rounds, messages, L0 recorded count, duration |
### Input formats
**Format A** — object wrapper:
```json
{
"sessions": [
{
"sessionKey": "user-alice",
"sessionId": "conv-001",
"conversations": [
[
{ "role": "user", "content": "Hello", "timestamp": 1711929600000 },
{ "role": "assistant", "content": "Hi!", "timestamp": 1711929601000 }
]
]
}
]
}
```
**Format B** — top-level array of sessions:
```json
[
{
"sessionKey": "user-alice",
"conversations": [
[
{ "role": "user", "content": "Hello" },
{ "role": "assistant", "content": "Hi!" }
]
]
}
]
```
| Field | Required | Notes |
|---|---|---|
| `sessionKey` | yes | Session identity |
| `sessionId` | no | Generated if omitted |
| `conversations` | yes | Array of rounds; each round is an array of messages |
| `role` | yes | Message role |
| `content` | yes | Message text |
| `timestamp` | no* | Epoch ms or ISO 8601; all-or-none across the file |
### Config override example
```json
{
"pipeline": {
"everyNConversations": 3,
"enableWarmup": false,
"l1IdleTimeoutSeconds": 2,
"l2DelayAfterL1Seconds": 1,
"l2MinIntervalSeconds": 1,
"l2MaxIntervalSeconds": 10
}
}
```
### Output layout
```text
/
├── conversations/ # L0 JSONL
├── records/ # L1 JSONL
├── scene_blocks/ # L2 scenes (if produced)
├── vectors.db # SQLite store (sqlite backend)
├── .metadata/
│ ├── manifest.json
│ └── checkpoint.json
└── .backup/
```
### Examples
```bash
openclaw memory-tdai seed --input conversations.json
openclaw memory-tdai seed --input data.json --output-dir ./seed-output
openclaw memory-tdai seed --input data.json --config seed-config.json --yes
openclaw memory-tdai seed --input data.json --strict-round-role --yes
```
For workflow detail (verification of L0→L1→L2→L3 on disk), see [Seed historical conversations](/seed-conversations).
---
## `migrate-sqlite-to-tcvdb`
Offline migration from local SQLite (`vectors.db`) into Tencent VectorDB (TCVDB). Optionally rewrites `openclaw.json` plugin config (`storeBackend: "tcvdb"`) and `.metadata/manifest.json`.
### Invoke
```bash
# After build (or from installed package with published dist)
migrate-sqlite-to-tcvdb [options]
npm run migrate-sqlite-to-tcvdb -- [options]
node ./bin/migrate-sqlite-to-tcvdb.mjs [options]
```
CLI entry: `scripts/migrate-sqlite-to-tcvdb/cli-entry.ts` → `runMigrationCli`. Stdout prints the migration summary as JSON; logs go to stderr with tag `[memory-tdai][migrate]`.
### Required options
Plugin data directory (for example `~/.openclaw/memory-tdai`).
Path to `openclaw.json`.
TCVDB HTTP base URL.
TCVDB username.
Target TCVDB database name.
Embedding model name recorded for the target store.
API key in clear text. Mutually exclusive with `--tcvdb-api-key-env`.
Environment variable name holding the API key. Mutually exclusive with `--tcvdb-api-key`. One of the two is required.
### Optional options
| Flag | Default | Description |
|---|---|---|
| `--sqlite-path` | `/vectors.db` | Source SQLite path |
| `--plugin-id` | `memory-tencentdb` | Plugin entry id written into `openclaw.json` |
| `--layers` | `l0,l1,l2,l3` | Comma-separated layer set |
| `--tcvdb-alias` | `""` | User alias |
| `--tcvdb-timeout-ms` | `10000` | Request timeout (ms) |
| `--tcvdb-ca-pem` | — | CA PEM path for HTTPS |
| `--bm25-language` | `zh` | `zh` or `en` |
| `--summary-json-path` | — | Write summary JSON to this path |
| `--job-id` | — | Optional job id for tracking |
| `--dry-run` | `false` | Preflight only; no writes |
| `--yes` | `false` | Skip interactive confirmation |
| `--no-apply-config` | apply on | Do not update `openclaw.json` |
| `--no-config-backup` | backup on | Skip config file backup before write |
| `--no-rewrite-manifest` | rewrite on | Do not rewrite manifest to `tcvdb` |
| `--no-fail-if-target-nonempty` | fail on | Allow non-empty target DB |
| `--no-verify-counts` | verify on | Skip post-migration count checks |
| `--no-bm25-enabled` | BM25 on | Disable BM25 sparse vectors |
| `-h, --help` | — | Print usage |
Boolean flags use Node `parseArgs` with `allowNegative: true` (so `--no-*` toggles the positive defaults).
### Config patch written on success
When `--apply-config` is true (default), the tool patches the plugin entry with:
```json
{
"storeBackend": "tcvdb",
"tcvdb": {
"url": "…",
"username": "…",
"apiKey": "…",
"database": "…",
"alias": "…",
"embeddingModel": "…",
"timeout": 10000
},
"bm25": {
"enabled": true,
"language": "zh"
}
}
```
### Empty source handling
If `plugin-data-dir` or `vectors.db` is missing, migration returns a zero-count summary (no hard failure)—useful for greenfield installs.
### Examples
```bash
# Dry-run preflight
migrate-sqlite-to-tcvdb \
--plugin-data-dir ~/.openclaw/memory-tdai \
--openclaw-config-path ~/.openclaw/openclaw.json \
--tcvdb-url http://127.0.0.1:80 \
--tcvdb-username root \
--tcvdb-api-key-env TCVDB_API_KEY \
--tcvdb-database agent_memory_prod \
--tcvdb-embedding-model bge-large-zh \
--dry-run
# Full migration
migrate-sqlite-to-tcvdb \
--plugin-data-dir ~/.openclaw/memory-tdai \
--openclaw-config-path ~/.openclaw/openclaw.json \
--tcvdb-url http://127.0.0.1:80 \
--tcvdb-username root \
--tcvdb-api-key-env TCVDB_API_KEY \
--tcvdb-database agent_memory_prod \
--tcvdb-embedding-model bge-large-zh \
--yes
# L1 only, no config rewrite
migrate-sqlite-to-tcvdb \
--plugin-data-dir ~/.openclaw/memory-tdai \
--openclaw-config-path ~/.openclaw/openclaw.json \
--tcvdb-url http://127.0.0.1:80 \
--tcvdb-username root \
--tcvdb-api-key-env TCVDB_API_KEY \
--tcvdb-database agent_memory_prod \
--tcvdb-embedding-model bge-large-zh \
--layers l1 \
--no-apply-config \
--no-rewrite-manifest \
--yes
```
See also [Migrate SQLite to Tencent VectorDB](/migrate-sqlite-tcvdb) and [Configure storage backends](/configure-storage).
---
## `export-tencent-vdb`
HTTP export client for **Tencent VectorDB only**. Connects with CLI-supplied credentials (no `.env` file), lists collections, and writes JSONL plus schema/meta sidecars.
### Invoke
```bash
export-tencent-vdb --url --username --api-key --database [options]
npm run export-tencent-vdb -- …
node ./bin/export-tencent-vdb.mjs …
```
### Connection parameters (required)
VDB instance HTTP base URL (for example `http://host:8100`).
Auth username (for example `root`).
Auth API key (sent as Bearer `account=…&api_key=…`).
Database name.
### Export options
| Flag | Default | Description |
|---|---|---|
| `--timeout` | `30000` | Per-request timeout (ms) |
| `-o, --output` | `./vdb-export-YYYY-MM-DD` | Output directory |
| `-c, --collection` | all | Export one collection (full name match) |
| `-f, --filter` | — | VDB filter expression |
| `-l, --limit` | all | Max documents (`>= 1`) |
| `--offset` | `0` | Start offset (`>= 0`; must be a multiple of page size `100`) |
| `--include-vectors` | off | Include dense `vector` field (skipped by default) |
| `--probe` | off | Connectivity check + list collections, then exit |
| `-h, --help` | — | Help |
`sparse_vector` (BM25) is always exported; only dense `vector` is gated by `--include-vectors`.
### Output layout
```text
/
├── .jsonl # one JSON document per line
├── schemas.json # collection schemas / embedding config
└── export-meta.json # export metadata
```
### Examples
```bash
export-tencent-vdb \
--url "http://gz-vdb-xxx:8100" --username root --api-key "xxx" --database mydb \
--probe
export-tencent-vdb \
--url "http://gz-vdb-xxx:8100" --username root --api-key "xxx" --database mydb
export-tencent-vdb \
--url "http://gz-vdb-xxx:8100" --username root --api-key "xxx" --database mydb \
-c mydb_l0_conversations -o /tmp/backup
export-tencent-vdb \
--url "http://gz-vdb-xxx:8100" --username root --api-key "xxx" --database mydb \
-f 'role = "user"' --include-vectors
```
---
## `read-local-memory`
Read-only inspector for a **local sqlite** memory data directory. L0/L1 read from `vectors.db` via `node:sqlite` with `PRAGMA query_only = ON`. L2 reads `scene_blocks/`; L3 reads `persona.md`.
### Invoke
```bash
read-local-memory -d [options]
npm run read-local-memory -- -d [options]
node ./bin/read-local-memory.mjs -d [options]
```
### Options
Local memory data directory; must exist. L0/L1 require `vectors.db` inside it.
Query one layer. Omit to scan all layers (overview).
Start time: ISO string or relative `7d` / `24h` / `30m` / `60s` (relative is “now minus N”).
End time (same formats as `--since`).
Page size (`>= 1`).
Page offset (`>= 0`).
Time sort: `desc` newest first, `asc` oldest first.
Column filters for L0/L1 SQLite tables. Operators: `=`, `!=`, `>`, `<`, `>=`, `<=`. Multiple conditions comma-separated. CamelCase field names map to snake_case columns.
Output format.
L2 single-file detail: return full content for one scene file name.
Print help and exit.
### Filter column allowlists
| Level | Allowed columns |
|---|---|
| L0 | `record_id`, `session_key`, `session_id`, `role`, `message_text`, `recorded_at`, `timestamp` |
| L1 | `record_id`, `content`, `type`, `priority`, `scene_name`, `session_key`, `session_id`, `timestamp_str`, `timestamp_start`, `timestamp_end`, `created_time`, `updated_time`, `metadata_json` |
Unknown filter columns fail fast with the allowed list printed.
### Examples
```bash
# All-layer overview
read-local-memory -d ~/.openclaw/memory-tdai
# L0 last 7 days
read-local-memory -d ~/.openclaw/memory-tdai -L L0 --since 7d
# L1 persona-type atoms
read-local-memory -d ~/.openclaw/memory-tdai -L L1 -f 'type=persona'
# Page 2 of L0 (20 per page)
read-local-memory -d ~/.openclaw/memory-tdai -L L0 -l 20 --offset 20
# Machine-readable
read-local-memory -d ~/.openclaw/memory-tdai -L L0 --since 7d --format json
```
`read-local-memory` targets the SQLite layout (`vectors.db`). It is not a TCVDB client; use `export-tencent-vdb` or Gateway search APIs for remote TCVDB inspection.
---
## Common failure modes
| Symptom | Likely cause | Probe |
|---|---|---|
| `precompiled artifact missing` / import fail | Script not built | `npm run build:scripts` then re-run bin |
| `openclaw memory-tdai` missing | Plugin not installed or CLI metadata not registered | `openclaw plugins list`; reinstall package |
| Seed aborts on non-empty output dir | Existing dir or checkpoint | New `--output-dir` or clean directory |
| Seed validation errors | Bad Format A/B or mixed timestamps | Fix JSON; use `--strict-round-role` only when every round is paired |
| Migrate: missing API key | Neither `--tcvdb-api-key` nor `--tcvdb-api-key-env` | Set one (not both) |
| Migrate: both key flags | Mutually exclusive | Keep a single key source |
| `read-local-memory`: no `vectors.db` | Wrong dir or tcvdb-only data | Point `-d` at sqlite data root |
| Export: missing required connection flags | Incomplete CLI | Pass `--url --username --api-key --database` |
---
## Related pages
Format A/B workflows, config overrides, and L0→L1→L2→L3 verification signals for seed runs.
Dry-run, layer selection, openclaw.json rewrites, and post-migration storeBackend checks.
sqlite vs tcvdb fields, embedding quadruplets, BM25 language, store health.
On-disk layout for OpenClaw memory-tdai, offload, and Hermes MEMORY_TENCENTDB_ROOT trees.
memory-tencentdb-ctl start/stop/status/health/logs and config subcommands.
Standalone Hermes sidecar endpoints including POST /seed and search routes.
Full plugin config schema used by seed --config merges and migrate rewrites.
Source-backed failure modes for plugin, embedding, offload, and Gateway recovery probes.
---
## 17. Data directories
> On-disk layout for OpenClaw (~/.openclaw/memory-tdai), offload (~/.openclaw/context-offload), and Hermes MEMORY_TENCENTDB_ROOT trees: conversations, records, scene_blocks, vectors.db, persona, checkpoints.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/17-data-directories.md
- Generated: 2026-07-09T07:35:44.368Z
### Source Files
- `index.ts`
- `src/utils/pipeline-factory.ts`
- `src/utils/manifest.ts`
- `src/utils/checkpoint.ts`
- `src/utils/openclaw-state-dir.ts`
- `scripts/README.memory-tencentdb-ctl.md`
- `src/offload/storage.ts`
---
title: "Data directories"
description: "On-disk layout for OpenClaw (~/.openclaw/memory-tdai), offload (~/.openclaw/context-offload), and Hermes MEMORY_TENCENTDB_ROOT trees: conversations, records, scene_blocks, vectors.db, persona, checkpoints."
---
TencentDB Agent Memory persists long-term memory and short-term context offload as separate directory trees. The OpenClaw plugin hard-codes the data folder name `memory-tdai` under the OpenClaw state dir (typically `~/.openclaw/memory-tdai`). Context offload defaults to `~/.openclaw/context-offload`. Standalone Gateway / Hermes mode defaults under `$MEMORY_TENCENTDB_ROOT` (typically `~/.memory-tencentdb/memory-tdai`). Package renames do **not** rename the on-disk data folder.
## Root paths
| Surface | Default root | Override | Owns |
| --- | --- | --- | --- |
| OpenClaw long-term memory | `~/.openclaw/memory-tdai` | `OPENCLAW_STATE_DIR` (or host `runtime.state.resolveStateDir()`) + fixed `memory-tdai` suffix | L0–L3 JSON/Markdown, SQLite `vectors.db`, checkpoints |
| Context offload | `~/.openclaw/context-offload` | `offload.dataDir` (absolute path) | Per-agent tool-pair offload, refs, Mermaid MMDs, `state.json` |
| Hermes / standalone Gateway | `~/.memory-tencentdb/memory-tdai` | `TDAI_DATA_DIR` or `MEMORY_TENCENTDB_ROOT` | Same memory tree as OpenClaw, plus `tdai-gateway.json` / logs |
| Hermes install code | `~/.memory-tencentdb/tdai-memory-openclaw-plugin` | `TDAI_INSTALL_DIR` | Gateway source + `node_modules` (not memory content) |
OpenClaw state-dir resolution order:
1. Host `runtime.state.resolveStateDir()`
2. `OPENCLAW_STATE_DIR`
3. `~/.openclaw`
Plugin data dir is always `path.join(stateDir, "memory-tdai")`.
Gateway / Hermes data-dir resolution order:
1. `TDAI_DATA_DIR` (or config `data.baseDir`, with `~/` expansion)
2. `$MEMORY_TENCENTDB_ROOT/memory-tdai` (default root `~/.memory-tencentdb`)
3. Legacy fallback `~/memory-tdai` when the new default does not exist yet
```text
OpenClaw host Hermes / standalone Gateway
───────────────── ───────────────────────────
$OPENCLAW_STATE_DIR $MEMORY_TENCENTDB_ROOT
└── memory-tdai/ (L0–L3) ├── memory-tdai/ ($TDAI_DATA_DIR)
└── context-offload/ (optional) │ ├── conversations/…
│ └── tdai-gateway.json
└── tdai-memory-openclaw-plugin/ (install)
```
The npm package may be `@tencentdb-agent-memory/memory-tencentdb`, but the OpenClaw data directory remains `memory-tdai`. Uninstalling the plugin does not delete that tree.
## Memory data tree (`memory-tdai`)
`initDataDirectories()` creates the core subdirectories on plugin (or Gateway) start:
:::files
~/.openclaw/memory-tdai/ # or $TDAI_DATA_DIR
├── conversations/ # L0 daily JSONL shards
│ └── YYYY-MM-DD.jsonl
├── records/ # L1 daily JSONL shards
│ └── YYYY-MM-DD.jsonl
├── scene_blocks/ # L2 scene Markdown files
│ └── *.md
├── persona.md # L3 persona (root file)
├── vectors.db # SQLite backend only (sqlite-vec + FTS5)
├── .metadata/
│ ├── manifest.json # store binding (sqlite|tcvdb)
│ ├── recall_checkpoint.json # L0/L1/L2/L3 pipeline cursors
│ ├── scene_index.json # engineering index of scene_blocks
│ └── instance_id # stable UUID for metrics
└── .backup/
├── persona/ # timestamped persona.md copies
└── scene_blocks/ # timestamped directory snapshots
:::
| Path | Layer | Format | Notes |
| --- | --- | --- | --- |
| `conversations/YYYY-MM-DD.jsonl` | L0 | One message per line | All sessions share the daily shard; `sessionKey` is a field, not the filename |
| `records/YYYY-MM-DD.jsonl` | L1 | One memory atom per line | Append-only; updates/merges append new lines and delete from the vector store |
| `scene_blocks/*.md` | L2 | Scene Markdown | LLM workspace is sandboxed to this directory during extraction |
| `persona.md` | L3 | Markdown | Written by persona generation; also receives scene navigation updates |
| `vectors.db` | L1 index | SQLite | Present when `storeBackend` is `sqlite` (default). Remote `tcvdb` does not create this file |
| `.metadata/*` | control plane | JSON / plain text | Manifest, checkpoint, scene index, instance id |
| `.backup/*` | recovery | files / dirs | Pre-LLM snapshots for persona and scene blocks |
### Layer file shapes
**L0 (`conversations/`)** — incremental capture from `agent_end`. Daily shard name uses the configured local timezone date. Each line is a flat message record (`sessionKey`, `sessionId`, `recordedAt`, `id`, `role`, `content`, `timestamp`).
**L1 (`records/`)** — extraction output, also daily-sharded. JSONL remains the durable local log; hybrid search/dedup prefers `vectors.db` or Tencent VectorDB when the store is healthy, and falls back to scanning JSONL when the store is degraded or missing.
**L2 (`scene_blocks/`)** — one `.md` file per scene. `scene_index.json` under `.metadata/` is maintained by the engineering side (not by the sandboxed LLM). Filenames are normalized after extraction so navigation links stay stable.
**L3 (`persona.md`)** — single file at the data-dir root. Before regeneration, the previous file is copied into `.backup/persona/` with a timestamped name.
### Store backend and `vectors.db`
| `storeBackend` | Local index file | Remote data |
| --- | --- | --- |
| `sqlite` (default) | `/vectors.db` | None |
| `tcvdb` | No local vector DB file | Collections in the configured Tencent VectorDB database |
`.metadata/manifest.json` records the first successful store binding:
- SQLite: `{ "type": "sqlite", "sqlite": { "path": "vectors.db" } }`
- TCVDB: `{ "type": "tcvdb", "tcvdb": { "url", "database", "alias?" } }`
Later config drift is logged at debug level; the manifest is not overwritten on every start. Seed runs may also set the `seed` field once.
### Checkpoints and metadata
| File | Role |
| --- | --- |
| `.metadata/recall_checkpoint.json` | Split-state checkpoint: global counters plus per-session `runner_states` (L0 capture cursor, L1 cursor, last scene name) and `pipeline_states` (conversation counts, extraction times, warm-up threshold) |
| `.metadata/scene_index.json` | Array of `{ filename, summary, heat, created, updated }` for fast L2 lookup |
| `.metadata/instance_id` | UUID created once; used in metric log lines |
| `.metadata/manifest.json` | Store binding + optional seed summary |
Checkpoint writes use temp-file + rename under a per-file async lock so concurrent `agent_end` / pipeline steps do not corrupt the file.
### Backups
`BackupManager` stores copies under `.backup//`:
| Category | When | Name pattern | Retention knobs |
| --- | --- | --- | --- |
| `persona` | Before L3 LLM write | `persona__offsetN.md` | `persona.backupCount` (default 3) |
| `scene_blocks` | Before L2 LLM extraction | `scene_blocks__offsetN/` (shallow file copy) | `persona.sceneBackupCount` |
If L2 extraction fails, the latest `scene_blocks` backup can be restored over the live directory.
### Local JSONL retention
Optional cleaner (`memoryCleanup.retentionDays` + `cleanTime`, default clean time `03:00`) deletes expired **daily shards** under `conversations/` and `records/` only. It does not remove `scene_blocks/`, `persona.md`, `vectors.db`, or `.metadata/`. Safety floors keep at least ~50 L0 and ~20 L1 records when counting would otherwise wipe the store.
## Context offload tree
Offload is independent of `memory-tdai`. Default root: `~/.openclaw/context-offload`, override with plugin config `offload.dataDir`.
:::files
~/.openclaw/context-offload/ # dataRoot
└── / # one dir per agent (from sessionKey)
├── offload-.jsonl # per-session tool-pair offload log
├── state.json # agent/session offload state
├── sessions-registry.json # sessionKey → real sessionId map
├── refs/ # full tool results as Markdown
│ └── .md
├── mmds/ # Mermaid canvas files
│ └── *.mmd
└── skills/ # optional skill artifacts from L4
└── /
:::
### Isolation rules
| Scope | Shared | Isolated |
| --- | --- | --- |
| Different agents | Separate under `dataRoot` | Entire agent directory |
| Same agent, different sessions | `mmds/`, `refs/`, `state.json` | `offload-.jsonl` |
| Multi-worker `swebench-wN` session keys | — | Worker suffix folded into `agentName` so each worker gets its own `dataDir` |
`sessionKey` format expected by offload: `agent::`. Path components are sanitized (unsafe characters → `_`).
### Key offload files
| Path | Purpose |
| --- | --- |
| `offload-*.jsonl` | Append-only offload entries (tool pairs, node ids, L1.5/L2 linkage) |
| `refs/*.md` | Full tool-result bodies; header includes tool name + timestamp |
| `mmds/*.mmd` | Symbolic Mermaid canvas used for injection |
| `state.json` | Persistent offload state for the agent directory |
| `sessions-registry.json` | Maps OpenClaw `sessionKey` to the real session id and offload filename |
Optional reclaimer (`offload.offloadRetentionDays`, minimum effective value 3) can expire old session JSONL files, orphan refs, and non-active MMDs under this tree. Debug logs under `dataRoot` can be size-capped via `offload.logMaxSizeMb` (default 50).
## Hermes and Gateway paths
Standalone / Hermes deployments consolidate under `MEMORY_TENCENTDB_ROOT` (default `~/.memory-tencentdb`):
:::files
~/.memory-tencentdb/
├── memory-tdai/ # $TDAI_DATA_DIR — same L0–L3 layout
│ ├── conversations/
│ ├── records/
│ ├── scene_blocks/
│ ├── persona.md
│ ├── vectors.db # if sqlite
│ ├── .metadata/
│ ├── .backup/
│ ├── tdai-gateway.json # or tdai-gateway.yaml (mode 0600 typical)
│ └── logs/ # standalone: gateway.stdout/stderr.log, gateway.pid
└── tdai-memory-openclaw-plugin/ # $TDAI_INSTALL_DIR — plugin source + node_modules
└── src/gateway/server.ts
:::
Additional Hermes-mode paths (outside the memory tree):
| Path | Mode | Role |
| --- | --- | --- |
| `$HERMES_HOME/logs/memory_tencentdb/` | hermes | Gateway stdout/stderr when supervised by Hermes |
| `$HERMES_HOME/env.d/memory-tencentdb-llm.sh` | hermes | LLM credentials for Gateway child process |
| `/etc/profile.d/memory-tencentdb-env.sh` | install | `MEMORY_TENCENTDB_GATEWAY_CMD` and related env |
Gateway config file search order:
1. `TDAI_GATEWAY_CONFIG`
2. `./tdai-gateway.yaml` or `./tdai-gateway.json` in CWD
3. `$TDAI_DATA_DIR/tdai-gateway.yaml` or `tdai-gateway.json`
4. Environment-only config
Legacy pre-0.4 locations `~/memory-tdai` and `~/tdai-memory-openclaw-plugin` are still detected; install scripts can migrate them into `$MEMORY_TENCENTDB_ROOT`.
## Seed output directories
`openclaw memory-tdai seed` writes a **separate** data tree (does not mutate the live plugin dir by default):
| Option | Result |
| --- | --- |
| `--output-dir ` | Explicit absolute/relative path |
| Default | `/memory-tdai-seed-` |
The seed runtime reuses `initDataDirectories` + the same L0/L1/L2/L3 writers, so the output tree matches the live layout (`conversations/`, `records/`, `scene_blocks/`, `persona.md`, `vectors.db`, `.metadata/`). Promote seed results into the live dir only with an explicit operational copy/merge after verification.
## Environment and config overrides
| Variable / config | Affects | Default |
| --- | --- | --- |
| `OPENCLAW_STATE_DIR` | OpenClaw state parent | `~/.openclaw` |
| (implicit) `memory-tdai` segment | Memory data folder name | hard-coded |
| `offload.dataDir` | Offload root | `~/.openclaw/context-offload` |
| `MEMORY_TENCENTDB_ROOT` | Hermes/standalone root | `~/.memory-tencentdb` |
| `TDAI_DATA_DIR` | Gateway data dir | `$MEMORY_TENCENTDB_ROOT/memory-tdai` |
| `TDAI_INSTALL_DIR` | Gateway install/code dir | `$MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin` |
| `TDAI_GATEWAY_CONFIG` | Gateway config file path | searched as above |
| `MEMORY_TENCENTDB_LOG_DIR` | Gateway log dir override | mode-dependent |
| `HERMES_HOME` | Hermes home | `~/.hermes` |
## Verify on disk
After a successful OpenClaw enable + gateway restart:
```bash
# Long-term memory
ls -la "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/"
ls "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/conversations/" 2>/dev/null
ls "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/records/" 2>/dev/null
test -f "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/.metadata/recall_checkpoint.json" && echo "checkpoint ok"
test -f "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/vectors.db" && echo "sqlite vectors present" || echo "no local vectors.db (tcvdb or not yet indexed)"
# Context offload (if enabled)
ls -la "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/context-offload/" 2>/dev/null
# Hermes / standalone
ls -la "${TDAI_DATA_DIR:-$HOME/.memory-tencentdb/memory-tdai}/"
```
Healthy signals:
- Subdirs `conversations`, `records`, `scene_blocks`, `.metadata`, `.backup` exist after first init
- L0 files appear after conversation capture; L1/L2/L3 appear after pipeline runs
- Gateway health does not require non-empty L1/L2; empty trees after fresh install are normal
- After package rename, the same `~/.openclaw/memory-tdai` tree continues to grow
## Related pages
L0–L3 producers, storage shapes, and pipeline triggers that fill this tree.
How tool-pair offload, Mermaid injection, and node_id drill-down use the offload tree.
sqlite vs tcvdb, embedding settings, and store health checks tied to vectors.db / remote DB.
Seed CLI flags and verification of a separate seed output directory.
memory-tencentdb-ctl paths, log locations, and TDAI_* env overrides.
Packages redacted config, logs, and the memory-tdai tree for support.
Preserve ~/.openclaw/memory-tdai when moving from @tdai/memory-tdai to the new package name.
---
## 18. Gateway control
> memory-tencentdb-ctl start/stop/status/health/logs and config subcommands in standalone vs hermes modes, path env overrides, and LLM credential injection.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/18-gateway-control.md
- Generated: 2026-07-09T07:35:55.970Z
### Source Files
- `scripts/memory-tencentdb-ctl.sh`
- `scripts/README.memory-tencentdb-ctl.md`
- `scripts/install_hermes_memory_tencentdb.sh`
- `src/gateway/server.ts`
- `src/gateway/config.ts`
---
title: "Gateway control"
description: "memory-tencentdb-ctl start/stop/status/health/logs and config subcommands in standalone vs hermes modes, path env overrides, and LLM credential injection."
---
`scripts/memory-tencentdb-ctl.sh` is the operations entry for the standalone TDAI Gateway HTTP sidecar (`src/gateway/server.ts`). It starts and stops the process, probes `GET /health`, tails logs, and merges LLM / embedding / Tencent VectorDB settings into `$TDAI_DATA_DIR/tdai-gateway.json` (mode `0600`, atomic write). Default mode is **standalone** (no writes under `~/.hermes`). Pass `--hermes` or set `MEMORY_TENCENTDB_MODE=hermes` to also write Hermes env fragments and expose `enable-hermes-memory`.
The script ships with the npm package under `scripts/` but is **not** registered in `package.json` `bin`. Call it by absolute path or symlink it into `PATH` yourself.
## Modes
| Mode | Activate | Lifecycle + `tdai-gateway.json` | Extra side effects |
|---|---|---|---|
| `standalone` (default) | No flag, or `--standalone` | Yes | Logs under `$TDAI_DATA_DIR/logs/`; does **not** touch `$HERMES_HOME` |
| `hermes` | `--hermes` or `MEMORY_TENCENTDB_MODE=hermes` | Yes | Logs under `$HERMES_HOME/logs/memory_tencentdb/`; `config llm` also writes `$HERMES_HOME/env.d/memory-tencentdb-llm.sh`; `enable-hermes-memory` allowed |
Hermes mode exists because Hermes supervisors spawn Gateway with `os.environ.copy()`. That child does not inherit a shell that already loaded `tdai-gateway.json` credentials, so LLM keys must also land in `$HERMES_HOME/env.d/*.sh` for Hermes to `source` before spawn. Embedding and VDB are **never** written to env files—only the JSON config.
```mermaid
flowchart TB
subgraph ctl ["memory-tencentdb-ctl"]
life["start / stop / restart / status / health / logs"]
cfg["config llm | embedding | vdb | vdb-off | show"]
enhm["enable-hermes-memory"]
end
subgraph disk ["On disk"]
json["$TDAI_DATA_DIR/tdai-gateway.json"]
logs_s["standalone: $TDAI_DATA_DIR/logs/"]
logs_h["hermes: $HERMES_HOME/logs/memory_tencentdb/"]
envd["hermes only: $HERMES_HOME/env.d/memory-tencentdb-llm.sh"]
hcfg["hermes only: $HERMES_HOME/config.yaml"]
end
subgraph process ["Gateway process"]
srv["src/gateway/server.ts → TdaiGateway"]
load["loadGatewayConfig()"]
end
life --> logs_s
life --> logs_h
life --> srv
cfg --> json
cfg -->|MODE=hermes and section=llm| envd
enhm --> hcfg
json --> load
envd -->|TDAI_LLM_* env override| load
load --> srv
```
## Paths and environment overrides
Canonical root (0.4.x+): `$MEMORY_TENCENTDB_ROOT` defaults to `~/.memory-tencentdb`.
| Variable | Default | Role |
|---|---|---|
| `MEMORY_TENCENTDB_ROOT` | `~/.memory-tencentdb` | Unified root for install + data |
| `TDAI_INSTALL_DIR` | `$MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin` | Plugin tree + `src/gateway/server.ts` + `node_modules` |
| `TDAI_DATA_DIR` | `$MEMORY_TENCENTDB_ROOT/memory-tdai` | Data + `tdai-gateway.json` |
| `HERMES_HOME` | `~/.hermes` | Hermes home (hermes mode only for writes) |
| `MEMORY_TENCENTDB_LOG_DIR` | mode-dependent (see below) | Override log/PID directory |
| `MEMORY_TENCENTDB_MODE` | `standalone` | Mode without CLI flags |
| `MEMORY_TENCENTDB_GATEWAY_HOST` | `127.0.0.1` | Host used by **ctl** for port listen / health |
| `MEMORY_TENCENTDB_GATEWAY_PORT` | `8420` | Port used by **ctl** for listen / health / stop |
| `MEMORY_TENCENTDB_GATEWAY_CMD` | (empty → fallback) | Full shell command to spawn Gateway |
Log directory after `_apply_mode_paths`:
- **standalone:** `$MEMORY_TENCENTDB_LOG_DIR` or `$TDAI_DATA_DIR/logs`
- **hermes:** `$MEMORY_TENCENTDB_LOG_DIR` or `$HERMES_HOME/logs/memory_tencentdb`
Files under the log dir: `gateway.pid`, `gateway.stdout.log`, `gateway.stderr.log`.
Legacy paths `~/tdai-memory-openclaw-plugin` and `~/memory-tdai` trigger WARN only; migration is owned by `install_hermes_memory_tencentdb.sh`, not the ctl.
### Gateway process bind vs ctl probe
Two env namespaces:
| Concern | Variables | Defaults |
|---|---|---|
| **ctl** status/health/stop by listening port | `MEMORY_TENCENTDB_GATEWAY_HOST` / `_PORT` | `127.0.0.1:8420` |
| **Node process** bind (`loadGatewayConfig`) | `TDAI_GATEWAY_HOST` / `TDAI_GATEWAY_PORT`, or `server.host` / `server.port` in the gateway config file | `127.0.0.1:8420` |
If you change only the ctl port variables, health checks track the new port while the process may still bind to 8420. Keep them aligned, or set `TDAI_GATEWAY_*` / `server.*` for the process and matching `MEMORY_TENCENTDB_GATEWAY_*` for the ctl.
### Config file resolution (Gateway process)
`loadGatewayConfig()` resolves config in order:
1. `TDAI_GATEWAY_CONFIG` (explicit path)
2. `./tdai-gateway.yaml` or `./tdai-gateway.json` in CWD
3. `/tdai-gateway.yaml` or `/tdai-gateway.json` (data dir from `TDAI_DATA_DIR` / defaults)
4. Environment-only defaults (no file)
Environment overrides beat file fields for LLM (`TDAI_LLM_BASE_URL`, `TDAI_LLM_API_KEY`, `TDAI_LLM_MODEL`, …) and server (`TDAI_GATEWAY_*`). Optional `TDAI_GATEWAY_API_KEY` enables Bearer auth on all routes except `GET /health`.
## Prerequisites
| Requirement | Notes |
|---|---|
| `bash` | Script shell |
| `python3` | Health check (`urllib`), JSON merge, hermes YAML edit |
| `node` ≥ 22 | Gateway runtime |
| `npx` | Fallback start command when `MEMORY_TENCENTDB_GATEWAY_CMD` unset |
| `lsof` or `ss` | Detect listening PID on Gateway port |
| `setsid` (optional) | Detach process group on start; falls back to `nohup` only |
Install first with `scripts/install_hermes_memory_tencentdb.sh` so `$TDAI_INSTALL_DIR` has sources and `node_modules`. The installer writes:
- `/etc/profile.d/memory-tencentdb-env.sh` — `MEMORY_TENCENTDB_GATEWAY_CMD`, host, port (interactive shells)
- `$HERMES_HOME/.env` — same keys for systemd-style Hermes that never sources `/etc/profile.d`
Install-time `GATEWAY_CMD` prefers absolute `node` + `--import tsx/esm` (PATH-independent). Ctl fallback (when `MEMORY_TENCENTDB_GATEWAY_CMD` is empty) is:
```bash
sh -c 'cd $TDAI_INSTALL_DIR && exec npx tsx src/gateway/server.ts'
```
## Invoke the script
Not a package `bin`. Examples:
```bash
# From npm install (local)
"$(npm root)/@tencentdb-agent-memory/memory-tencentdb/scripts/memory-tencentdb-ctl.sh" --help
# Global npm install
"$(npm root -g)/@tencentdb-agent-memory/memory-tencentdb/scripts/memory-tencentdb-ctl.sh" --help
# Symlink for ops
sudo ln -sf "$SCRIPT" /usr/local/bin/memory-tencentdb-ctl
```
Global flags (any position before the subcommand in the flag-strip pass):
| Flag | Effect |
|---|---|
| `--hermes` | `MODE=hermes` |
| `--standalone` | `MODE=standalone` |
| `--dry-run` | Print planned writes / kills; no disk or process mutation |
| `-h` / `--help` | Usage |
## Lifecycle commands
Commands that spawn Gateway call `source_user_envs` first:
1. Always (if readable): `/etc/profile.d/memory-tencentdb-env.sh`
2. Hermes mode only: `/etc/profile.d/hermes-env.sh` and every `$HERMES_HOME/env.d/*.sh`
### `start`
1. Create log + data dirs (`ensure_paths`).
2. If something already listens on `$GATEWAY_PORT`, log WARN and return 0.
3. Resolve start command (`MEMORY_TENCENTDB_GATEWAY_CMD` or `npx tsx` fallback).
4. Spawn with `setsid nohup … &` (or `nohup` only), write shell wrapper PID to `gateway.pid`.
5. Poll up to ~15s for port listen + `GET /health`; WARN + exit 1 if still unhealthy.
### `stop`
1. Resolve PIDs via `lsof`/`ss` on `$GATEWAY_PORT`.
2. `SIGTERM`; after ~5s still listening → `SIGKILL`.
3. Remove `gateway.pid` (also tries leftover wrapper PID if no listener).
### `restart`
`stop` (ignore failure) → short sleep → `start`.
### `status`
Prints mode, host:port, install/data/log paths, config file existence, RUNNING/STOPPED, and health OK/UNHEALTHY. Hermes mode also prints `memory.provider` from `$HERMES_HOME/config.yaml` and lists `$HERMES_HOME/env.d/*.sh`.
### `health`
Python3 `GET http://$GATEWAY_HOST:$GATEWAY_PORT/health` (timeout 3s). Exit 0 + log healthy, or exit 1.
### `logs [out|err|all] [N]`
`tail -n N -f` on stdout, stderr, or both. Defaults: `all`, `N=200`.
```bash
memory-tencentdb-ctl start
memory-tencentdb-ctl status
memory-tencentdb-ctl health
memory-tencentdb-ctl logs err 500
memory-tencentdb-ctl stop
```
## Config subcommands
All merges target `$TDAI_DATA_DIR/tdai-gateway.json` with mode `0600` via temp file + rename.
JSON layout written by the ctl:
| Subcommand | JSON path |
|---|---|
| `config llm` | top-level `llm.{baseUrl,apiKey,model}` |
| `config embedding` | `memory.embedding.{…}` |
| `config vdb` | `memory.tcvdb.{…}` and usually `memory.storeBackend` |
| `config vdb-off` | `memory.storeBackend = "sqlite"` (± drop `memory.tcvdb`) |
Optional `--restart` on write commands runs `cmd_restart` after a successful write.
### `config llm`
Required: `--api-key`, `--base-url` (`http://` or `https://`), `--model`.
**Both modes:** merge into `tdai-gateway.json` `$.llm`.
**Hermes only:** also write `$HERMES_HOME/env.d/memory-tencentdb-llm.sh` (`0600`):
```bash
export TDAI_LLM_BASE_URL=...
export TDAI_LLM_API_KEY=...
export TDAI_LLM_MODEL=...
export MEMORY_TENCENTDB_LLM_BASE_URL="$TDAI_LLM_BASE_URL"
export MEMORY_TENCENTDB_LLM_API_KEY="$TDAI_LLM_API_KEY"
export MEMORY_TENCENTDB_LLM_MODEL="$TDAI_LLM_MODEL"
```
`TDAI_LLM_*` match Gateway `loadGatewayConfig` env overrides. `MEMORY_TENCENTDB_LLM_*` aliases are for the Hermes Python provider schema.
```bash
# standalone — JSON only
memory-tencentdb-ctl config llm \
--api-key "sk-..." \
--base-url "https://api.openai.com/v1" \
--model "gpt-4o" \
--restart
# hermes — JSON + env.d
memory-tencentdb-ctl --hermes config llm \
--api-key "sk-..." \
--base-url "https://api.openai.com/v1" \
--model "gpt-4o" \
--restart
```
### `config embedding`
| Flag | Required | Notes |
|---|---|---|
| `--provider` | yes | `none` disables vectors; else e.g. `openai`, `deepseek`, `qclaw`, … |
| `--api-key` | yes if not `none` | |
| `--base-url` | yes if not `none` | must be `http(s)://` |
| `--model` | yes if not `none` | |
| `--dimensions` | yes if not `none` | positive integer |
| `--proxy-url` | if `provider=qclaw` | |
| `--restart` | no | |
`provider=none` writes `{"provider":"none","enabled":false}` only. Non-`none` sets `enabled: true` plus credentials.
```bash
memory-tencentdb-ctl config embedding \
--provider openai \
--api-key "sk-..." \
--base-url "https://api.openai.com/v1" \
--model "text-embedding-3-small" \
--dimensions 1536 \
--restart
memory-tencentdb-ctl config embedding --provider none --restart
```
### `config vdb`
| Flag | Required | Notes |
|---|---|---|
| `--url` | yes | `http(s)://` |
| `--api-key` | yes | |
| `--database` | yes | |
| `--username` | no | default `root` |
| `--alias` | no | |
| `--ca-pem` | no | path must be readable; not copied |
| `--embedding-model` | no | |
| `--no-set-backend` | no | keep current `storeBackend` |
| `--restart` | no | |
Default also sets `memory.storeBackend` to `"tcvdb"`.
### `config vdb-off`
Sets `memory.storeBackend` to `"sqlite"`.
| Flag | Effect |
|---|---|
| (default) | Keep `memory.tcvdb` credentials |
| `--purge-creds` | Delete entire `memory.tcvdb` object |
| `--restart` | Restart Gateway |
Does **not** change Hermes `memory.provider`. If the JSON file is missing, writes a minimal `{"memory":{"storeBackend":"sqlite"}}`.
There is no zero-arg `vdb-on`: re-run full `config vdb` with required fields to switch back.
### `config show`
Prints `tdai-gateway.json` with `apiKey` / `password` / `token` redacted as ``. Hermes mode also prints `memory-tencentdb-*.sh` env files with keys partially redacted.
## Hermes-only: `enable-hermes-memory`
Requires `MODE=hermes`. Idempotently sets `$HERMES_HOME/config.yaml` → `memory.provider: memory_tencentdb` without rewriting the whole file:
1. Prefer `ruamel.yaml` round-trip (comments/order/quotes preserved) if installed.
2. Else minimal in-place line edit of the existing `provider:` line (sibling indent copied).
3. If no `memory:` section, append a minimal block.
```bash
memory-tencentdb-ctl --hermes enable-hermes-memory
source "$HERMES_HOME/env.d/memory-tencentdb-llm.sh"
# then restart hermes per your host process manager
```
Standalone mode: command exits with code 1.
## Typical workflows
```bash
bash scripts/install_hermes_memory_tencentdb.sh
memory-tencentdb-ctl config llm \
--api-key "sk-..." --base-url "https://api.openai.com/v1" --model gpt-4o
memory-tencentdb-ctl config embedding \
--provider openai --api-key "sk-..." --base-url "https://api.openai.com/v1" \
--model text-embedding-3-small --dimensions 1536
# optional remote store:
# memory-tencentdb-ctl config vdb --url "http://...:8100" --api-key "..." --database openclaw_memory
memory-tencentdb-ctl start
memory-tencentdb-ctl health # expect JSON body with status ok
```
```bash
bash scripts/install_hermes_memory_tencentdb.sh
export MEMORY_TENCENTDB_MODE=hermes # optional: skip --hermes each time
memory-tencentdb-ctl config llm \
--api-key "sk-..." --base-url "https://api.openai.com/v1" --model gpt-4o
memory-tencentdb-ctl config embedding \
--provider openai --api-key "sk-..." --base-url "https://api.openai.com/v1" \
--model text-embedding-3-small --dimensions 1536
memory-tencentdb-ctl start # manual fallback; Hermes supervisor may own this
memory-tencentdb-ctl enable-hermes-memory
source "$HERMES_HOME/env.d/memory-tencentdb-llm.sh"
# restart hermes agent/gateway per your install
```
```bash
# Keep tcvdb creds, local store only
memory-tencentdb-ctl config vdb-off --restart
# Later: re-declare full vdb (required flags always validated)
memory-tencentdb-ctl config vdb \
--url "http://...:8100" --api-key "..." --database openclaw_memory --restart
# Or purge creds permanently
memory-tencentdb-ctl config vdb-off --purge-creds --restart
```
## Exit codes
| Code | Meaning |
|---|---|
| `0` | Success |
| `1` | Bad args / validation (e.g. non-http base URL; hermes-only command in standalone) |
| `2` | Write failure (disk/permissions) — reserved for write helpers |
| `127` | Missing dependency (`python3` / `node` / `npx`) |
## Security and ops notes
- Sensitive files (`tdai-gateway.json`, `env.d/memory-tencentdb-llm.sh`) are `0600`. Env files contain plaintext API keys—do not commit them.
- Default Gateway bind is loopback. Without `TDAI_GATEWAY_API_KEY` / `server.apiKey`, routes other than `/health` are unauthenticated; the process logs a WARN at startup (louder if non-loopback).
- `--dry-run` previews config and kill actions without applying them.
- Foreground debug: `cd "$TDAI_INSTALL_DIR" && npx tsx src/gateway/server.ts` (or the installer’s absolute `node --import tsx/esm` command).
- Port conflict example that moves **ctl** probes: `MEMORY_TENCENTDB_GATEWAY_PORT=18420 memory-tencentdb-ctl restart` — also set `TDAI_GATEWAY_PORT=18420` (or `server.port`) so the Node process binds the same port.
- Optional systemd: `Type=forking` around `memory-tencentdb-ctl start` / `stop` (stateless HTTP sidecar; no special readiness protocol).
## Verification
| Check | Expected signal |
|---|---|
| `memory-tencentdb-ctl status` | `state: RUNNING`, paths match your env overrides |
| `memory-tencentdb-ctl health` | HTTP body from `/health`, exit 0 |
| `memory-tencentdb-ctl config show` | Redacted JSON; hermes mode also lists env.d |
| Hermes env inheritance | `env.d/memory-tencentdb-llm.sh` present after `config llm --hermes` |
| Hermes provider | `enable-hermes-memory` → `memory.provider = memory_tencentdb` in `status` |
## Related pages
Deploy plugin tree, Gateway CMD env files, and Hermes provider packaging.
Endpoint contracts for /health, /recall, /capture, search, session, and seed.
StandaloneHostAdapter and TdaiCore boundary used by the Gateway process.
MEMORY_TENCENTDB_ROOT / memory-tdai layout for logs, vectors, and persona.
sqlite vs tcvdb fields and embedding provider quadruplets.
Gateway circuit breaker, silent failures, and recovery probes.
---
## 19. Export diagnostics
> Run export-diagnostic.sh to package redacted config, gateway logs, and memory-tdai data; privacy notes and local-only delivery expectations.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/19-export-diagnostics.md
- Generated: 2026-07-09T07:35:56.996Z
### Source Files
- `scripts/export-diagnostic.sh`
- `SKILL-DIAGNOSTIC-EXPORT.md`
- `scripts/bugfix-20260423/BUGFIX-20260423-SOP.md`
- `src/utils/openclaw-state-dir.ts`
---
title: "Export diagnostics"
description: "Run export-diagnostic.sh to package redacted config, gateway logs, and memory-tdai data; privacy notes and local-only delivery expectations."
---
`scripts/export-diagnostic.sh` packages a local OpenClaw + memory-tencentdb support bundle: environment inventory, gateway and rolling logs, the full `memory-tdai` data tree, a redacted `openclaw.json`, and installed plugin metadata. The script writes only under a user-chosen (or default) local path, creates `openclaw-diagnostic-.tar.gz`, and never uploads the archive.
The archive is **local-only**. There is no network transfer step. Review contents before sharing. `memory-tdai/` includes raw conversation text and other high-sensitivity memory data even when config secrets are redacted.
## Scope and constraints
| Item | Behavior |
|------|----------|
| Entry point | `bash scripts/export-diagnostic.sh [output_base_dir]` |
| Package surface | Not a `package.json` `bin`; run from a repo checkout that contains `scripts/export-diagnostic.sh` |
| Host focus | OpenClaw state directory and its `memory-tdai` plugin data |
| Not collected | `context-offload/`, Hermes/`MEMORY_TENCENTDB_ROOT` trees (`~/.memory-tencentdb/…`), remote VectorDB contents |
| Delivery | Manual only (copy/send archive after local review) |
Plugin package rename does not change the data path: `@tencentdb-agent-memory/memory-tencentdb` still stores under `/memory-tdai` (hardcoded in the plugin and the export script). Log tags remain `[memory-tdai]`.
## Prerequisites
- Bash with `set -euo pipefail` (script uses strict mode)
- Readable OpenClaw state directory (see [State directory resolution](#state-directory-resolution))
- `tar` for packaging
- `node` recommended for config redaction (falls back to rough `grep`/`sed` redaction if Node fails)
- Optional: `openclaw` or `pnpm openclaw` for version capture in `env-info.txt`
- Optional: `json5` available to Node for non-strict JSON configs; otherwise plain `JSON.parse`
## State directory resolution
The script picks `STATE_DIR` in this order:
1. `OPENCLAW_STATE_DIR` if set
2. `$HOME/.openclaw` if that directory exists
3. `$HOME/.clawdbot` if that directory exists
4. Otherwise exit `1` with: OpenClaw work directory not found
Plugin runtime resolution (`resolveOpenClawStateDir`) prefers host `runtime.state.resolveStateDir()`, then `OPENCLAW_STATE_DIR`, then `~/.openclaw`. The export script additionally accepts legacy `~/.clawdbot` when present.
Plugin data path used by both runtime and export:
```text
$STATE_DIR/memory-tdai
```
Typical default: `~/.openclaw/memory-tdai`.
## Run the export
```bash
OPENCLAW_DIR="${OPENCLAW_STATE_DIR:-$HOME/.openclaw}"
[ -d "$OPENCLAW_DIR" ] || OPENCLAW_DIR="$HOME/.clawdbot"
ls -la "$OPENCLAW_DIR/" 2>/dev/null
ls -la "$OPENCLAW_DIR/memory-tdai/" 2>/dev/null
```
```bash
# Default output base: ~/Downloads
bash scripts/export-diagnostic.sh
# Custom output base directory
bash scripts/export-diagnostic.sh /tmp
```
The script prints the archive path and size. Confirm:
- `openclaw-diagnostic-.tar.gz` exists under the output base
- Unpacked directory sibling: `openclaw-diagnostic-/`
- Expected members listed below are present (missing optional trees are non-fatal)
Inspect `memory-tdai/` and logs. Redacted config is not a full privacy guarantee for conversation content. Share only what you intend to send.
### CLI parameters
Directory that will contain `openclaw-diagnostic-/` and the sibling `.tar.gz`. Pass as the first positional argument.
### Environment inputs
| Variable | Effect |
|----------|--------|
| `OPENCLAW_STATE_DIR` | Forces the OpenClaw state root used for logs, config, extensions, and `memory-tdai` |
### Success output
On success the script prints a summary similar to:
```text
═══════════════════════════════════════════════════
✅ 诊断数据导出完成
═══════════════════════════════════════════════════
📦 压缩包: /openclaw-diagnostic-.tar.gz
📏 大小:
包含内容:
- env-info.txt
- logs/
- memory-tdai/
- openclaw-config-redacted.json
- plugins-info.txt
⚠️ 安全提示:
- 配置文件已自动脱敏（API Key、Token、Password 等已移除）
- models/secrets/channels/env 等敏感配置块已整体替换
- 记忆数据中可能包含用户对话内容，请确认后再发送
📤 请手动检查后发送给研发团队
═══════════════════════════════════════════════════
```
Failure modes:
| Condition | Result |
|-----------|--------|
| No `~/.openclaw`, `~/.clawdbot`, or `OPENCLAW_STATE_DIR` | Exit `1` before packaging |
| Missing `memory-tdai` | Warning; export continues without that tree |
| Missing `openclaw.json` | Warning; no config artifact |
| Node redaction error | Falls back to line-based `grep`/`sed` redaction |
| Missing optional logs / subdirs | Skipped with `|| true` style handling |
## Archive layout
:::files
openclaw-diagnostic-<timestamp>/
├── env-info.txt
├── openclaw-config-redacted.json
├── plugins-info.txt
├── logs/
│ ├── gateway-logs/ # copy of $STATE_DIR/logs/
│ └── rolling-logs/ # up to 3 newest /tmp/openclaw/openclaw-*.log (tail 5000 lines each)
└── memory-tdai/
├── conversations/ # L0 JSONL
├── records/ # L1 JSONL
├── scene_blocks/ # L2 Markdown
├── persona.md # L3 persona (if present)
├── vectors.db # SQLite vector + FTS store (if present)
├── .metadata/ # checkpoints, scene_index, etc.
└── .backup/ # optional; may be large
:::
### What each artifact is for
| Path in archive | Source | Privacy | Use in triage |
|-----------------|--------|---------|---------------|
| `env-info.txt` | Live probe | Low | OS, Node/pnpm, OpenClaw version, `STATE_DIR`, recursive `memory-tdai` listing, `du` sizes |
| `logs/gateway-logs/` | `$STATE_DIR/logs/` | Low–medium | Gateway stdout/stderr, config audit, command logs if present |
| `logs/rolling-logs/` | `/tmp/openclaw/openclaw-*.log` | Low–medium | Recent JSON Lines rolling logs (newest 3 files, last 5000 lines each) |
| `memory-tdai/` | `$STATE_DIR/memory-tdai/` | **High** | L0–L3 data, SQLite index, checkpoints, backups |
| `openclaw-config-redacted.json` | `$STATE_DIR/openclaw.json` after redact | Low | Plugin entries, store/embedding shape without secrets |
| `plugins-info.txt` | `$STATE_DIR/extensions/*` | Low | Installed extension dirs and package name/version when available |
### Log sources (OpenClaw)
| Kind | Typical path | Export behavior |
|------|--------------|-----------------|
| Gateway daemon logs | `$STATE_DIR/logs/` (e.g. `gateway.log`, `gateway.err.log`) | Full tree copy into `logs/gateway-logs/` |
| Rolling JSONL logs | `/tmp/openclaw/openclaw-YYYY-MM-DD.log` | Newest 3 files; each truncated to last 5000 lines |
| Config audit / commands | under `$STATE_DIR/logs/` | Included if present in that tree |
### memory-tdai subtrees copied
| Subpath | Layer / role |
|---------|----------------|
| `conversations/` | L0 raw conversations (daily JSONL) |
| `records/` | L1 structured memories (daily JSONL) |
| `scene_blocks/` | L2 scene Markdown |
| `persona.md` | L3 persona file |
| `.metadata/` | Checkpoints and indexes (e.g. recall checkpoint) |
| `vectors.db` | Local SQLite vector + FTS index |
| `.backup/` | Rolling backups (optional; can dominate size) |
## Config redaction
Redaction targets `$STATE_DIR/openclaw.json` and writes `openclaw-config-redacted.json`.
| Rule | Replacement |
|------|-------------|
| Key matches `api_?key\|token\|password\|secret\|credential` (case-insensitive) and value is a non-empty string | `***REDACTED(chars)***` (empty string stays empty) |
| SecretRef-like object with `source`, `id`, and `provider` | Same shape with `id: "***REDACTED***"` |
| Top-level keys `models`, `secrets`, `channels`, `env` | `***REDACTED_SECTION(use openclaw config get to inspect)***` |
| Under path `gateway.auth`, keys matching `token\|password` | `***REDACTED***` for strings |
| Other fields (including most of `plugins`) | Kept after recursive walk so non-secret plugin settings remain for debugging |
Recursive redaction still strips nested `apiKey` / `token` / `password` / `secret` / `credential` string fields inside `plugins`. Provider, model, enable flags, and similar non-matching keys remain.
If the Node path fails, the fallback strips lines that look like long secret string assignments and attempts a coarse section rewrite for `models` / `secrets` / `channels` / `env`. Prefer fixing Node rather than relying on the fallback before sharing.
## Privacy and delivery expectations
| Class | Handling |
|-------|----------|
| API keys, tokens, passwords, secrets, SecretRefs | Redacted in config export |
| Top-level models / secrets / channels / env | Section redacted |
| Plugin non-secret config | Retained for diagnosis |
| L0 conversations and other memory files | **Not redacted** — full local copy |
| Upload / telemetry | **None** — script only writes local files and `tar.gz` |
Recommended delivery workflow:
1. Keep the archive on the machine that produced it.
2. Unzip and spot-check `memory-tdai/conversations/`, `records/`, and logs for content you do not want shared.
3. Optionally remove `.backup/` or other large/sensitive subtrees and re-tar.
4. Send the archive through your approved channel (chat, email, ticket). The repository does not define an automated upload endpoint.
## Manual export fallback
When the script cannot run (for example Node redaction path unavailable and you prefer full control):
```bash
EXPORT_DIR=~/Downloads/openclaw-diagnostic-$(date +%Y%m%d-%H%M%S)
mkdir -p "$EXPORT_DIR"
cp -r "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/logs/" "$EXPORT_DIR/logs/" 2>/dev/null || true
cp /tmp/openclaw/openclaw-$(date +%Y-%m-%d).log "$EXPORT_DIR/" 2>/dev/null || true
cp -r "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/memory-tdai/" "$EXPORT_DIR/memory-tdai/" 2>/dev/null || true
# Config must be redacted by hand before sharing
cp "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/openclaw.json" \
"$EXPORT_DIR/openclaw-config-NEEDS-MANUAL-REDACTION.json"
cd "$(dirname "$EXPORT_DIR")"
tar -czf "${EXPORT_DIR}.tar.gz" "$(basename "$EXPORT_DIR")"
```
Manual config copies are **not** redacted. Remove `models` / `secrets` / `channels` / `env` blocks and all key/token values before sending.
## Agent skill hook
`SKILL-DIAGNOSTIC-EXPORT.md` defines skill `openclaw-diagnostic-export` for agents that should run this workflow when users ask to export diagnostics, package on-site data, or collect logs for support. The skill mirrors the script’s steps, privacy warnings, and post-export handoff guidance.
## Triage signals inside the bundle
| Question | Where to look |
|----------|----------------|
| Did the plugin load? | `logs/` → search `[memory-tdai]` |
| Is recall running? | `logs/` → search `[recall]` |
| Did L1/L2/L3 pipeline fire? | `logs/` → search `[pipeline]` |
| Embedding / store backend shape? | `openclaw-config-redacted.json` → `plugins.entries` |
| Disk pressure / empty store? | `env-info.txt` (`du`, listings) |
| Checkpoint / cursor state? | `memory-tdai/.metadata/` (e.g. recall checkpoint files) |
## Out of scope
- **Hermes / standalone Gateway data** under `MEMORY_TENCENTDB_ROOT` (default `~/.memory-tencentdb/memory-tdai`) is not collected by this script. Use host-specific log and data paths for those deployments.
- **Context offload** data under `context-offload/` is not part of the archive.
- **Tencent VectorDB remote collections** are not dumped; only local `vectors.db` when present under OpenClaw `memory-tdai`.
- **Automatic ticket upload** is not implemented.
## Related pages
On-disk layout for `memory-tdai`, offload, and Hermes roots that the export may partially sample.
Source-backed failure modes and recovery probes that diagnostics packages support.
Full plugin config schema fields retained (non-secret) in redacted exports.
`memory-tencentdb-ctl` start/stop/status/logs for live probes alongside offline exports.
Why the data dir remains `memory-tdai` after migrating to `@tencentdb-agent-memory/memory-tencentdb`.
Other package bins and `openclaw memory-tdai` commands (export-diagnostic is a repo script, not a published bin).
---
## 20. Troubleshooting
> Source-backed failure modes: plugin silent, no recall, embedding degrade, aggressive cleanup, offload slot/patch missing, Gateway circuit breaker, and recovery probes.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/20-troubleshooting.md
- Generated: 2026-07-09T07:36:47.867Z
### Source Files
- `SKILL.md`
- `src/config.ts`
- `src/core/hooks/auto-recall.ts`
- `src/core/store/embedding.ts`
- `src/utils/ensure-hook-policy.ts`
- `hermes-plugin/memory/memory_tencentdb/client.py`
- `hermes-plugin/memory/memory_tencentdb/tests/test_memory_tencentdb_recovery.py`
- `scripts/bugfix-20260423/BUGFIX-20260423-SOP.md`
---
title: "Troubleshooting"
description: "Source-backed failure modes: plugin silent, no recall, embedding degrade, aggressive cleanup, offload slot/patch missing, Gateway circuit breaker, and recovery probes."
---
Failure modes for **TencentDB Agent Memory** (`@tencentdb-agent-memory/memory-tencentdb`, plugin id `memory-tencentdb`) surface on three runtime paths: the OpenClaw plugin (`index.ts` / `TdaiCore`), the context-offload slot (`offload.enabled` + `plugins.slots.contextEngine`), and the Hermes Gateway sidecar (`MemoryTencentdbProvider` + HTTP client on port `8420`). Logs use the historical tag `[memory-tdai]`; on-disk data stays under `memory-tdai` even after the package rename.
## Symptom matrix
| Symptom | Likely surface | First probe |
| --- | --- | --- |
| No `[memory-tdai]` logs after install | Plugin not enabled / Gateway not restarted | `openclaw.json` + gateway restart |
| Capture/hooks never fire on OC ≥ 2026.4.23 | `hooks.allowConversationAccess` blocked | Policy field + optional bugfix script |
| Turns run but no memory injection | Recall path | `recall.*`, score threshold, strategy fallback |
| Only keyword hits / no vectors | Embedding quadruplet or store degrade | `[EMBEDDING CONFIG ERROR]`, `vectors.db` |
| History vanishes after a few days | Retention / aggressive cleanup | `l0l1RetentionDays`, `allowAggressiveCleanup` |
| Offload never compresses tool output | Slot or `after-tool-call` patch | `contextEngine` + patch script |
| Hermes tools empty / “circuit breaker tripped” | Gateway sidecar | `/health`, breaker cooldown, watchdog |
## Plugin silent (no logs, no capture)
### Enablement and restart
Plugin registration always parses `api.pluginConfig` and logs at `[memory-tdai]`. If that tag never appears:
1. Confirm install: `openclaw plugins install @tencentdb-agent-memory/memory-tencentdb` (or `plugins update memory-tencentdb`).
2. Enable in `~/.openclaw/openclaw.json` (or `OPENCLAW_CONFIG_PATH`):
```json
{
"memory-tencentdb": {
"enabled": true
}
}
```
3. Restart: `openclaw gateway restart`.
4. Expect log prefix `[memory-tdai]` and data under `{OPENCLAW_STATE_DIR or ~/.openclaw}/memory-tdai/` (`conversations/`, `records/`, `scene_blocks/`, `vectors.db`).
Zero-config defaults leave `capture`, `extraction`, and `recall` enabled; `offload.enabled` defaults to `false`.
### Silent hook block (`allowConversationAccess`)
On OpenClaw **v2026.4.23+**, non-bundled plugins without `hooks.allowConversationAccess: true` can have conversation hooks (`agent_end`, related session hooks) blocked while the plugin still “loads.”
Auto-patch (`ensurePluginHookPolicy`):
- Runs only on gateway-start processes (skips `plugins install|doctor|config|…`).
- Applies only when host version ≥ **2026.4.24** (`HOOK_POLICY_MIN_VERSION`); unparsable/missing `api.runtime.version` → **no** auto-patch (safe for older hosts).
- Writes:
```json
{
"plugins": {
"entries": {
"memory-tencentdb": {
"hooks": {
"allowConversationAccess": true
}
}
}
}
}
```
- Prefers SDK `mutateConfigFile` with `afterWrite: { mode: "restart" }`; falls back to a manual write and warns that **restart is required**.
- If config uses `$include`, logs a manual-edit warning instead of rewriting.
**OpenClaw 2026.4.23 only:** Zod `.strict()` rejected the field (Issue #73806). Use the repo SOP:
```bash
openclaw gateway stop
bash scripts/bugfix-20260423/bugfix-20260423.sh
# verify allowConversationAccess under plugins.entries.memory-tencentdb.hooks
# verify dist schema contains allowConversationAccess once
openclaw gateway run
```
### Verification signals
| Signal | Meaning |
| --- | --- |
| `[memory-tdai] [hook-policy] ✅ …` | Policy patched |
| `[memory-tdai] Config parsed: capture=…, recall=…` | Plugin registered |
| Data dir created under `memory-tdai/` | Init path ran |
| No logs after restart | Wrong config path / plugin not loaded / host not restarted |
## No recall
Auto-recall registers only when `recall.enabled` is true (`before_prompt_build` → `core.handleBeforeRecall` / `performAutoRecall`).
### Checklist
| Check | Default / rule | Failure effect |
| --- | --- | --- |
| `recall.enabled` | `true` | Hook never registered |
| User prompt text | Non-empty | Skip L1 search (persona/scene may still inject) |
| Sanitized query length | ≥ 2 chars after `sanitizeText` | Empty search result |
| `recall.scoreThreshold` | `0.3` | Hits below threshold dropped |
| `recall.maxResults` | `5` | Caps injected lines |
| `recall.timeoutMs` | `5000` | Timeout → **skip injection** (warn), non-blocking |
| `recall.strategy` | `hybrid` | If embedding unavailable → **keyword** fallback |
| Session filter | `capture.excludeAgents` | Matched agents skip recall |
| Exceptions | — | Logged + `error_degradation` → `degradedTo: "no_recall"` |
Log patterns:
- `Recall complete (…ms), no context to inject` — ran, nothing to inject (threshold, empty index, or budget).
- `⚠️ Recall timed out after …ms — skipping memory injection` — protect latency over recall.
- `Strategy "hybrid"|"embedding" requested but EmbeddingService not available, falling back to keyword` — vector path offline.
- Keyword path needs FTS5; if FTS unavailable, keyword returns empty (no full-table O(N) fallback).
Smoke: multi-turn with memorable facts, then call tools `tdai_memory_search` / `tdai_conversation_search` and re-open a session to observe injection.
## Embedding degrade
### Config-time disable (non-throwing)
`parseConfig` does **not** fail the plugin when remote embedding is incomplete. It sets `embedding.configError`, forces `embedding.enabled = false`, and logs at register time:
`[memory-tdai] [EMBEDDING CONFIG ERROR] …`
| `embedding.provider` | Behavior |
| --- | --- |
| `"none"` (default) | Vectors off; `dimensions` forced to `0` (no vec0 tables until a real provider is set) |
| `"local"` | Treated as disabled at config entry; `configError` tells you to use a remote provider |
| Remote (`openai`, `deepseek`, …) | Requires **apiKey + baseUrl + model + dimensions** (all present, dimensions > 0). Any missing field → disabled + detailed `Missing: …` |
| `"qclaw"` | Same quadruplet **plus** `proxyUrl` |
`sendDimensions` defaults to `true` (OpenAI Matryoshka). Self-hosted models that reject `dimensions` (e.g. BGE-M3 HTTP 400) need `"sendDimensions": false`.
### Runtime store degrade
`VectorStore` (`sqlite`) can enter `degraded = true` if sqlite-vec load/init fails. Then upserts/searches become safe no-ops; logs include `VectorStore entering degraded mode` / `[L1-search] SKIPPED (degraded mode)`. Gateway `GET /health` reports:
```json
{
"status": "ok" | "degraded",
"stores": {
"vectorStore": true | false,
"embeddingService": true | false
}
}
```
`status` is `degraded` when `core.getVectorStore()` is missing. `TdaiCore` continues with JSONL / non-vector paths when store init fails.
### Factory behavior
SQLite path creates a remote `EmbeddingService` only when `embedding.enabled && provider !== "local" && apiKey`. Otherwise vector search is unavailable and hybrid/embedding strategies fall back to keyword.
## Aggressive cleanup
### Retention gate
From `capture.l0l1RetentionDays` + `capture.allowAggressiveCleanup` (default `false`):
| `l0l1RetentionDays` | `allowAggressiveCleanup` | Effective cleaner |
| --- | --- | --- |
| `≤ 0` | any | Disabled (`retentionDays` undefined) |
| `≥ 3` | any | Enabled with that many days |
| `1` or `2` | `false` | **Forced off** (treated as invalid/dangerous) |
| `1` or `2` | `true` | Enabled (aggressive) |
Daily schedule uses `capture.cleanTime` (default `"03:00"`).
### Safety guards (even when enabled)
| Guard | Value | Effect |
| --- | --- | --- |
| Min retain L0 | 50 | Skip L0 delete if total ≤ 50 |
| Min retain L1 | 20 | Skip L1 delete if total ≤ 20 |
| SQLite ratio block | > 80% of rows would expire in one pass | Log `BLOCKED: would delete … exceeds 80% safety threshold`, delete **0** |
| Degraded store | `isDegraded()` | `deleteExpired*` skipped |
If history “disappears,” verify you did not set retention to 1–2 with `allowAggressiveCleanup: true`, and inspect cleaner logs: `[memory-tdai][cleaner]`, `cleaner_summary`, `BLOCKED`.
## Offload slot / patch missing
Context offload is independent of long-term memory defaults (`offload.enabled` default `false`).
### Required wiring
1. **Config**
```json
{
"memory-tencentdb": {
"config": {
"offload": {
"enabled": true
}
}
},
"plugins": {
"slots": {
"contextEngine": "memory-tencentdb"
}
}
}
```
Without `plugins.slots.contextEngine = "memory-tencentdb"`, OpenClaw will not route context-engine work to this plugin.
2. **Runtime patch** — `scripts/openclaw-after-tool-call-messages.patch.sh` (also best-effort `postinstall` in `package.json`). Idempotent; re-run after OpenClaw upgrades. Without it, after-tool-call message offload/recovery is incomplete.
3. **Helper** — `scripts/setup-offload.sh` applies the patch, sets the slot, and toggles `offload.enabled`. Patch failure **aborts** enable.
4. **Data** — default root `~/.openclaw/context-offload` (override with `offload.dataDir`).
If `cfg.offload.enabled` is true, `registerOffload` runs; failures are logged as `Offload module registration failed` without taking down memory.
## Gateway circuit breaker (Hermes)
Hermes provider `MemoryTencentdbProvider` talks to the Node Gateway sidecar via `MemoryTencentdbSdkClient` (default `http://127.0.0.1:8420`, timeout 10s).
### Breaker and recovery constants
| Constant | Value | Role |
| --- | --- | --- |
| `_BREAKER_THRESHOLD` | 5 | Consecutive failures trip breaker |
| `_BREAKER_COOLDOWN_SECS` | 60 | Pause business calls while open |
| `_RECOVER_COOLDOWN_SECS` | 15 | Throttle request-path `ensure_running` |
| `_WATCHDOG_INTERVAL_SECS` | 10 | Background health / resurrect |
On trip: log `memory-tencentdb circuit breaker tripped after N failures. Pausing for 60s.` Tool path may return JSON error `memory-tencentdb Gateway temporarily unavailable (circuit breaker open).`
### Recovery model
```text
Request path Background
──────────── ──────────
prefetch / capture / tools
│
├─ _ensure_alive_for_request()
│ if available → proceed
│ if breaker open → empty/disable (no recover storm)
│ else → _try_recover_gateway() (15s throttle)
│
└─ on HTTP errors → _record_failure()
after 5 → open breaker 60s
Watchdog every 10s:
process alive + available → noop
/health ok but available=false → reattach client, clear breaker
down → _try_recover_gateway(bypass_cooldown=True)
```
Known deadlock class (covered by tests): `_gateway_available` stuck `False` while every request short-circuited before the failure path. Mitigations: **lazy probe** on request guards + **watchdog** even with no traffic.
Auth: client may send `Authorization: Bearer …` from `MEMORY_TENCENTDB_GATEWAY_API_KEY` or `TDAI_GATEWAY_API_KEY`. Mismatch with Gateway secret → HTTP failures that count toward the breaker.
### Hermes-side symptoms
| Message / behavior | Action |
| --- | --- |
| Gateway not available at startup | Set `MEMORY_TENCENTDB_GATEWAY_CMD` or ensure auto-discovery finds `src/gateway/server.ts`; check `~/.hermes/logs/memory_tencentdb/gateway.stderr.log` |
| Search tools missing from LLM | `get_tool_schemas()` is `[]` until Gateway reachable **or** `MEMORY_TENCENTDB_GATEWAY_CMD` / `PORT` set |
| Circuit breaker tripped | Inspect sidecar health/logs; wait cooldown or fix root cause so watchdog can clear |
| Capture backlog | ≥ 4 in-flight sync threads — Gateway slow/hung |
## Recovery probes
Use these as ordered ops checks, not product APIs.
### OpenClaw plugin
```bash
openclaw --version # need >= 2026.3.13
node -v # need >= 22.16.0
openclaw gateway restart
# Config
# - memory-tencentdb enabled
# - plugins.entries.memory-tencentdb.hooks.allowConversationAccess == true (OC >= 2026.4.23/24)
# Logs: [memory-tdai], [hook-policy], [EMBEDDING CONFIG ERROR], before_prompt_build Recall complete
# Data: $OPENCLAW_STATE_DIR/memory-tdai or ~/.openclaw/memory-tdai
```
### Offload
```bash
bash scripts/openclaw-after-tool-call-messages.patch.sh
# or: bash scripts/setup-offload.sh
# Confirm plugins.slots.contextEngine == memory-tencentdb and offload.enabled
# Data: ~/.openclaw/context-offload
```
### Gateway / Hermes
```bash
# Standalone or hermes mode (see gateway-control page for full flags)
memory-tencentdb-ctl status
memory-tencentdb-ctl health # GET /health → status ok|degraded
# Env pins
# MEMORY_TENCENTDB_GATEWAY_CMD, MEMORY_TENCENTDB_GATEWAY_HOST, MEMORY_TENCENTDB_GATEWAY_PORT
# MEMORY_TENCENTDB_GATEWAY_API_KEY / TDAI_GATEWAY_API_KEY
# MEMORY_TENCENTDB_LOG_DIR
```
`GET /health` response shape: `status`, `version`, `uptime`, `stores.vectorStore`, `stores.embeddingService`.
### Package diagnostics
```bash
bash scripts/export-diagnostic.sh [output-dir]
```
Packages redacted config, gateway logs, and `memory-tdai` data for local-only analysis.
### Automated recovery tests (Hermes)
`hermes-plugin/memory/memory_tencentdb/tests/test_memory_tencentdb_recovery.py` covers:
- Watchdog start, death detection, resurrect, external restart reattach
- Circuit breaker reset on recovery
- Lazy probe self-heal on `prefetch` / tools / `sync_turn`
- Open breaker still short-circuits without respawn storm
## Related pages
Prerequisites, install/update, postinstall patch behavior.
Zero-config enable, restart, success signals, smoke tools.
offload.enabled, contextEngine slot, after-tool-call patch.
sqlite vs tcvdb, embedding quadruplets, store health.
Full schema: recall, embedding, capture retention, offload.
memory-tencentdb-ctl start/stop/status/health/logs.
/health, /recall, /capture, search and seed endpoints.
export-diagnostic.sh package layout and privacy notes.
Provider install, sidecar supervision, config keys.
memory-tdai, context-offload, Hermes MEMORY_TENCENTDB_ROOT trees.
---
## 21. Contributing
> Dev setup with openclaw plugins install --link, test scripts (vitest), build targets, commit conventions, and PR expectations.
- Page Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/pages/21-contributing.md
- Generated: 2026-07-09T07:36:27.265Z
### Source Files
- `CONTRIBUTING.md`
- `package.json`
- `vitest.config.ts`
- `vitest.e2e.config.ts`
- `.github/workflows/pr-ci.yml`
- `CHANGELOG.md`
---
title: "Contributing"
description: "Dev setup with openclaw plugins install --link, test scripts (vitest), build targets, commit conventions, and PR expectations."
---
Local development for `@tencentdb-agent-memory/memory-tencentdb` is a **link-from-source** OpenClaw plugin workflow: install dependencies, register the repo with `openclaw plugins install --link .`, edit TypeScript under `index.ts` / `src/`, and restart the Gateway. Runtime loads `.ts` directly (Node ≥ 22.16.0 type stripping); packaging and package bins use separate `npm run build*` targets. Pull requests target `main`, require DCO `Signed-off-by`, and are gated by `.github/workflows/pr-ci.yml` (install, pack, manifest, size).
## Prerequisites
| Requirement | Constraint |
|-------------|------------|
| Node.js | `>=22.16.0` (`package.json` `engines`) |
| Package manager | `npm` or `pnpm` |
| OpenClaw (for plugin host loop) | `>=2026.3.13` (CONTRIBUTING; plugin `openclaw.compat.pluginApi` / `minGatewayVersion`) |
| Peer (optional) | `openclaw` `>=2026.3.7`, `node-llama-cpp` `^3.16.2` |
CI runners use Node **22** and `npm install --ignore-scripts`.
## Develop from source
OpenClaw loads plugin TypeScript from the linked directory. No plugin rebuild is required between edits.
```bash
git clone https://github.com/Tencent/TencentDB-Agent-Memory.git
cd TencentDB-Agent-Memory
npm install
```
```bash
openclaw plugins install --link .
```
`install --link` registers the **current directory** as a local plugin. Source changes apply after a Gateway restart.
1. Edit `index.ts`, `src/**`, `openclaw.plugin.json`, or `hermes-plugin/` as needed.
2. Restart the OpenClaw Gateway.
3. Confirm load via Gateway / plugin logs tagged `[memory-tdai]` and your usual smoke path (`tdai_memory_search` / capture).
`postinstall` runs `scripts/openclaw-after-tool-call-messages.patch.sh` (errors swallowed). Local `npm install` may attempt that host patch; CI deliberately uses `--ignore-scripts`.
### Repository layout (contribution hotspots)
```text
├── index.ts # OpenClaw plugin shell (tools, hooks, TdaiCore wiring)
├── openclaw.plugin.json # Plugin id, contracts, configSchema
├── package.json # Scripts, engines, openclaw metadata, bins
├── tsdown.config.ts # Plugin ESM bundle → dist/
├── vitest.config.ts # Unit tests
├── vitest.e2e.config.ts # E2E include pattern
├── src/
│ ├── config.ts
│ ├── core/ # Host-neutral memory (conversation, record, scene, persona, store, …)
│ ├── adapters/ # openclaw / standalone host adapters
│ ├── gateway/ # Standalone HTTP gateway
│ ├── offload/ # Context-engine offload
│ ├── cli/ # openclaw memory-tdai CLI registration
│ └── utils/
├── hermes-plugin/ # Hermes MemoryProvider adapter
├── scripts/ # ctl, migrate, export, patch, hermes install
├── bin/ # Published CLI wrappers (need build:scripts)
├── CONTRIBUTING.md
└── .github/ # PR/issue templates, pr-ci.yml
```
CONTRIBUTING’s older flat `src/conversation` tree maps to `src/core/*` in the current tree; prefer paths above when placing PRs.
## npm scripts
| Script | Command surface | Purpose |
|--------|-----------------|---------|
| `test` | `vitest run` | Unit suite (default config) |
| `test:watch` | `vitest` | Watch mode |
| `test:coverage` | `vitest run --coverage` | v8 coverage (`text` / `html` / `lcov`) |
| `build` | `build:plugin` + `build:scripts` | Full package + CLI compile |
| `build:plugin` | `tsdown` | Bundle `index.ts` → `dist/index.mjs` |
| `build:scripts` | three `tsc` projects | migrate / export / read-local-memory |
| `build:migrate-sqlite-to-vdb` | `tsc -p scripts/migrate-sqlite-to-tcvdb/tsconfig.json` | Migration CLI |
| `build:export-tencent-vdb` | `tsc --project scripts/export-tencent-vdb/tsconfig.json` | Export CLI |
| `build:read-local-memory` | `tsc --project scripts/read-local-memory/tsconfig.json` | Local memory reader |
| `migrate-sqlite-to-tcvdb` | `node ./bin/migrate-sqlite-to-tcvdb.mjs` | Run migration bin |
| `export-tencent-vdb` | `node ./bin/export-tencent-vdb.mjs` | Run export bin |
| `read-local-memory` | `node ./bin/read-local-memory.mjs` | Run reader bin |
| `prepack` | `npm run build` | Ensures build before pack |
| `postinstall` | patch script `\|\| true` | Optional OpenClaw after-tool-call patch |
Published bins:
- `migrate-sqlite-to-tcvdb`
- `export-tencent-vdb`
- `read-local-memory`
Each `bin/*.mjs` wrapper imports precompiled output under `scripts/*/dist/`. Build the matching script target before invoking a bin against a dirty tree.
### Build target: plugin (`tsdown`)
`tsdown.config.ts`:
| Option | Value |
|--------|--------|
| Entry | `./index.ts` |
| Out dir | `./dist` |
| Format | ESM |
| Platform | node |
| DTS / sourcemap | off |
| Externals | `openclaw` (+ subpaths), `node:*`, all `dependencies` / `peerDependencies` / `optionalDependencies` |
`package.json` `main` / `exports` point at `./dist/index.mjs`. Link-dev OpenClaw uses `openclaw.extensions: ["./index.ts"]`, not the packed `dist/` path.
### When you need `npm run build`
| Workflow | Build needed? |
|----------|----------------|
| OpenClaw `--link` edit/reload | No (source `.ts`) |
| `npm pack` / publish | Yes (`prepack` → `build`) |
| CLI bins / migrate-export tools | Yes (`build:scripts` or full `build`) |
| PR CI pack job | Yes (pack triggers `prepack` → build when scripts run) |
## Tests (Vitest)
### Unit tests (`vitest.config.ts`)
| Setting | Value |
|---------|--------|
| Environment | `node` |
| Pool | `forks` |
| Include | `src/**/*.test.ts`, `__tests__/**/*.test.ts` |
| Exclude | `dist/**`, `node_modules/**`, `**/*.e2e.test.ts` |
| Timeouts | `testTimeout` / `hookTimeout` **120_000** ms |
| Mocks | `clearMocks`, `restoreMocks`, `unstubEnvs`, `unstubGlobals` |
| Coverage | provider `v8`; include `src/**/*.ts`, `index.ts`; exclude tests/dist/node_modules |
```bash
npm test
npm run test:watch
npm run test:coverage
```
Existing co-located unit examples include `src/utils/time.test.ts`, `src/utils/sanitize.test.ts`, `src/utils/no-think-fetch.test.ts`, `src/offload/auth-profile-key.test.ts`. Prefer the same `*.test.ts` placement next to the module under test.
### E2E config (`vitest.e2e.config.ts`)
| Setting | Value |
|---------|--------|
| Include | `**/*.e2e.test.ts` |
| Exclude | `dist/**`, `node_modules/**` |
| `testTimeout` | 120_000 ms |
| `hookTimeout` | 60_000 ms |
There is **no** `package.json` script that selects this config. Run explicitly when adding e2e files:
```bash
npx vitest run --config vitest.e2e.config.ts
```
`.npmignore` / `package.json` `files` exclude tests from the published tarball (`!src/**/*.test.ts`, `!src/**/__tests__/`, etc.).
## Pull request CI
Workflow: `.github/workflows/pr-ci.yml`
Triggers: `pull_request` → branch **`main`**
Concurrency: `ci-${{ github.ref }}` with `cancel-in-progress: true`
| Job | Depends on | What it checks |
|-----|------------|----------------|
| **Install** | — | Checkout, Node 22, cache `node_modules` by `hashFiles('package.json')`, `npm install --ignore-scripts` on cache miss |
| **Pack** | install | Restore cache → `npm pack --dry-run` → `npm pack` → upload `*.tgz` (7-day retention) |
| **Manifest** | — (parallel) | `openclaw.plugin.json` exists + valid JSON; required `id`; optional `configSchema` object; `package.json` `openclaw.extensions`, `openclaw.compat.pluginApi`, `openclaw.build.openclawVersion` |
| **Size Guard** | pack | Tarball size ≤ **2048 KB** |
CI does **not** currently run `vitest`. Local `npm test` (and targeted builds) remain the contributor self-test bar from the PR template.
## Commit conventions
Format from `CONTRIBUTING.md`:
```text
():
Closes #123
Signed-off-by: Your Name
```
### Types (aligned with PR template change types)
| Type | Meaning | PR Change Type checkbox |
|------|---------|-------------------------|
| `fix` | Bug fix | Bug fix |
| `feat` | New feature | New feature |
| `docs` | Documentation | Documentation update |
| `perf` | Performance | Code optimization |
| `refactor` | No behavior change | Code optimization |
| `test` | Tests | — |
| `chore` | Build / tooling / deps | — |
### Scope examples
`store`, `hooks`, `persona`, `scene`, `record`, `conversation`, `gateway`, `hermes`
### Code style expectations
- Match existing TypeScript style.
- Prefer English identifiers.
- Comment **why** at non-obvious logic.
- Import order: Node built-ins → third-party → internal modules.
## DCO (required)
Every commit must include a `Signed-off-by` line (Developer Certificate of Origin). Commits without it are not merged.
```bash
git commit -s -m "feat(store): add batch insert support"
```
## Pull request expectations
1. Fork and branch from **`main`** (default PR target).
2. Keep commits focused/atomic; use the type/scope convention above.
3. Self-test: `npm test` (and `npm run build` / relevant script builds if you touch packaging or bins).
4. Update user-facing docs (`README.md`, `CHANGELOG.md`, `openclaw.plugin.json` descriptions) when behavior or config changes.
5. Open a PR using `.github/PULL_REQUEST_TEMPLATE.md`.
### PR template fields
| Section | Expectation |
|---------|-------------|
| Description | What the PR does |
| Related Issue | e.g. `Fix #123` |
| Change Type | Bug fix / New feature / Documentation update / Code optimization |
| Self-test checklist | Verified locally; no existing features affected |
| Additional notes | Optional |
### Issue templates
| Template | Title prefix | Typical fields |
|----------|--------------|----------------|
| Bug report | `[Bug]` | OpenClaw version, plugin version, OS, repro, expected, logs |
| Feature request | `[Feature]` | Problem, desired solution, alternatives |
| Question | (consultation) | Question body per template |
## Security and license
| Topic | Rule |
|-------|------|
| Security vulnerabilities | Email **agentmemory@tencent.com** (do not open a public issue for sensitive reports) |
| Contribution license | MIT (`LICENSE`); submitting a PR accepts MIT licensing of the contribution |
| Changelog style | Keep a Changelog + SemVer (`CHANGELOG.md`) |
## Verification checklist before open PR
```bash
# Unit tests
npm test
# Optional coverage
npm run test:coverage
# If changing pack surface / bins
npm run build
npm pack --dry-run
# If changing plugin manifest or package openclaw metadata
# (mirrors CI Manifest job)
node -e "JSON.parse(require('fs').readFileSync('openclaw.plugin.json','utf8'))"
```
Confirm Gateway still loads the linked plugin after restart and that new config keys (if any) appear in `openclaw.plugin.json` `configSchema`.
## Related pages
Prerequisites, openclaw plugins install/update, link-from-source install, postinstall patch behavior.
Package bins and seed CLI, including build prerequisites for migrate/export/read tools.
Source-backed failure modes useful when a local link install or Gateway reload misbehaves.
Product surface and host entry points (OpenClaw plugin, Hermes, Gateway).
---