# Migrate SQLite to Tencent VectorDB

> Build and run migrate-sqlite-to-tcvdb: dry-run, layer selection, openclaw.json rewrites, and post-migration storeBackend=tcvdb checks.

- Repository: TencentCloud/TencentDB-Agent-Memory
- GitHub: https://github.com/TencentCloud/TencentDB-Agent-Memory
- Human docs: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97
- Complete Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/llms-full.txt

## Source Files

- `scripts/migrate-sqlite-to-tcvdb/README.md`
- `scripts/migrate-sqlite-to-tcvdb/cli-entry.ts`
- `scripts/migrate-sqlite-to-tcvdb/sqlite-to-tcvdb.ts`
- `scripts/migrate-sqlite-to-tcvdb/config-write.ts`
- `bin/migrate-sqlite-to-tcvdb.mjs`
- `SKILL-MIGRATION.md`

---

---
title: "Migrate SQLite to Tencent VectorDB"
description: "Build and run migrate-sqlite-to-tcvdb: dry-run, layer selection, openclaw.json rewrites, and post-migration storeBackend=tcvdb checks."
---

`migrate-sqlite-to-tcvdb` is the offline CLI that copies local memory data from SQLite (`vectors.db` via `VectorStore`) into Tencent Cloud VectorDB (`TcvdbMemoryStore`), then optionally rewrites OpenClaw plugin config to `storeBackend: "tcvdb"` and updates `<plugin-data-dir>/.metadata/manifest.json`. The package bin is `migrate-sqlite-to-tcvdb` (`bin/migrate-sqlite-to-tcvdb.mjs` → compiled `cli-entry.js`). Logs go to stderr under `[memory-tdai][migrate]` / `[memory-tdai][migrate-cli]`; the process prints a JSON migration summary on stdout.

<Warning>
This is **storage backend migration** (sqlite → tcvdb). It is not the package rename path from `@tdai/memory-tdai` to `@tencentdb-agent-memory/memory-tencentdb`. For package rename, see the package-rename docs page.
</Warning>

## Prerequisites

| Requirement | Detail |
|---|---|
| Node.js | `>= 22.16.0` (package `engines`) |
| Built migration artifacts | `npm run build:migrate-sqlite-to-vdb` (note the script name uses `vdb`) |
| Plugin data dir | Typically `~/.openclaw/memory-tdai` (shared data path; not deleted by plugin uninstall) |
| OpenClaw config | Single-file JSON/JSON5 path (default consumer: `~/.openclaw/openclaw.json`) |
| TCVDB endpoint | Reachable URL, username, database, embedding model, and API key |
| Embedding model | Must match the TCVDB collection / server-side embedding model you intend to use |

:::files
scripts/migrate-sqlite-to-tcvdb/
├── cli-entry.ts          # CLI entry; prints JSON summary
├── sqlite-to-tcvdb.ts    # parse args, preflight, migrate, verify
├── config-write.ts       # openclaw.json storeBackend=tcvdb patch
├── manifest-write.ts     # .metadata/manifest.json → type tcvdb
├── tsconfig.json
└── dist/                 # tsc output (gitignored)
bin/migrate-sqlite-to-tcvdb.mjs
:::

## Build and invoke

Compile before the first run (or after script source changes):

```bash
npm run build:migrate-sqlite-to-vdb
```

Output lands under `scripts/migrate-sqlite-to-tcvdb/dist/` (entry: `dist/scripts/migrate-sqlite-to-tcvdb/cli-entry.js`).

Run via npm script, package bin, or node:

<CodeGroup>
```bash title="npm script"
npm run migrate-sqlite-to-tcvdb -- \
  --plugin-data-dir ~/.openclaw/memory-tdai \
  --openclaw-config-path ~/.openclaw/openclaw.json \
  --tcvdb-url http://127.0.0.1:80 \
  --tcvdb-username root \
  --tcvdb-api-key-env TCVDB_API_KEY \
  --tcvdb-database agent_memory_prod \
  --tcvdb-embedding-model bge-large-zh \
  --dry-run
```

```bash title="package bin (after install)"
migrate-sqlite-to-tcvdb \
  --plugin-data-dir ~/.openclaw/memory-tdai \
  --openclaw-config-path ~/.openclaw/openclaw.json \
  --tcvdb-url http://127.0.0.1:80 \
  --tcvdb-username root \
  --tcvdb-api-key-env TCVDB_API_KEY \
  --tcvdb-database agent_memory_prod \
  --tcvdb-embedding-model bge-large-zh \
  --yes
```
</CodeGroup>

`package.json` registers:

- **bin:** `migrate-sqlite-to-tcvdb`
- **scripts:** `build:migrate-sqlite-to-vdb`, `migrate-sqlite-to-tcvdb`

<Note>
Use `npm run migrate-sqlite-to-tcvdb` (hyphenated). There is no `migrate:sqlite-to-tcvdb` script in `package.json`.
</Note>

## Migration flow

```mermaid
flowchart LR
  subgraph source [Local source]
    SQLite["vectors.db<br/>VectorStore L0/L1"]
    Files["plugin-data-dir<br/>scene_blocks + persona.md"]
    Manifest[".metadata/manifest.json"]
  end
  subgraph cli [migrate-sqlite-to-tcvdb]
    Preflight["collectMigrationPreflight"]
    Migrate["page upsert L1/L0<br/>syncProfiles"]
    Verify["count verify + 10s delay"]
    Config["writeMigrationPluginConfig"]
    ManWrite["rewriteMigrationManifest"]
  end
  subgraph target [TCVDB + OpenClaw]
    TCVDB["TcvdbMemoryStore"]
    OC["openclaw.json<br/>storeBackend=tcvdb"]
  end
  SQLite --> Preflight
  Files --> Preflight
  Manifest --> Preflight
  Preflight -->|dry-run| Summary["JSON summary stdout"]
  Preflight -->|live| Migrate
  Migrate --> TCVDB
  Migrate --> Verify
  Verify --> Config
  Config --> OC
  Verify --> ManWrite
  ManWrite --> Manifest
```

| Phase | Behavior |
|---|---|
| Preflight | Opens SQLite; counts L0/L1; lists local profiles; reads manifest store type |
| Dry-run | Returns preflight summary only; **no** data, config, or manifest writes |
| Data migrate | Paged upserts (page size **50**): L1 → L0 → profiles (when layers allow) |
| Target guard | Default: abort if target L0/L1/profile counts are non-zero |
| Verify | Default: wait **10s**, then require target counts to equal source counts |
| Config | Default: merge `storeBackend`, `tcvdb`, `bm25` into `plugins.entries.<plugin-id>.config` |
| Manifest | Default: set store binding to `type: "tcvdb"`; backup existing to `manifest.json.migrate.bak` |

**Empty / missing source handling**

- Missing `plugin-data-dir` or missing SQLite file → empty source counts, no throw (fresh install path).
- Source counts all zero → skip data migration; still apply config/manifest when not dry-run.

**What is not re-uploaded**

- Local sqlite-vec embedding blobs are not copied. `TcvdbMemoryStore` uses **server-side** embedding on upsert; migration passes records without client embeddings.
- On-disk JSONL conversation/record files stay in place; the CLI migrates SQLite-indexed L0/L1 rows and local profile files into TCVDB profile sync.

## CLI options

### Required

<ParamField body="plugin-data-dir" type="path" required>
Plugin data directory (e.g. `~/.openclaw/memory-tdai`).
</ParamField>

<ParamField body="openclaw-config-path" type="path" required>
Path to a single-file OpenClaw config (JSON/JSON5). Writer fails if the file cannot be parsed as such.
</ParamField>

<ParamField body="tcvdb-url" type="string" required>
TCVDB service URL.
</ParamField>

<ParamField body="tcvdb-username" type="string" required>
TCVDB username.
</ParamField>

<ParamField body="tcvdb-database" type="string" required>
TCVDB database name.
</ParamField>

<ParamField body="tcvdb-embedding-model" type="string" required>
Server-side embedding model name (e.g. `bge-large-zh`).
</ParamField>

API key — **exactly one** of:

| Flag | Role |
|---|---|
| `--tcvdb-api-key <key>` | Plaintext key |
| `--tcvdb-api-key-env <VAR>` | Read key from environment variable `VAR` |

Providing both throws. Missing both throws. Empty env value throws.

### Optional paths and identity

| Flag | Default | Notes |
|---|---|---|
| `--sqlite-path` | `<plugin-data-dir>/vectors.db` | Alternate snapshot DB |
| `--plugin-id` | `memory-tencentdb` | Key under `plugins.entries` when writing config |
| `--tcvdb-alias` | `""` | Optional alias stored in config + manifest |
| `--tcvdb-timeout-ms` | `10000` | Must be finite and `> 0` |
| `--tcvdb-ca-pem` | — | CA PEM path for HTTPS clients |
| `--summary-json-path` | — | Also write summary JSON to this file |
| `--job-id` | — | Accepted for tracing; stored on resolved options |

### Layer selection

| Flag | Default | Values |
|---|---|---|
| `--layers` | `l0,l1,l2,l3` | Comma-separated subset of `l0`, `l1`, `l2`, `l3` |

| Layer token | Migrates |
|---|---|
| `l1` | L1 memory rows from SQLite → `upsertL1` / `upsertL1Batch` |
| `l0` | L0 conversation rows from SQLite → `upsertL0` / `upsertL0Batch` |
| `l2` **or** `l3` | Local profiles via `listLocalProfiles` + `syncProfiles` |

Profile discovery (when `l2` or `l3` is selected):

- `scene_blocks/*.md` → profile type `l2`
- non-empty `persona.md` (navigation stripped) → profile type `l3`

Selecting only `l2` or only `l3` still runs the **same** profile sync (both L2 blocks and L3 persona if present). Omitting both skips profiles entirely.

### Boolean flags (defaults true unless noted)

Node `parseArgs` supports negation (`allowNegative: true`):

| Flag | Default | Effect |
|---|---|---|
| `--dry-run` | `false` | Preflight only; no writes |
| `--yes` | `false` | Recorded in summary; **no interactive prompt is implemented** — live runs proceed without a confirm gate |
| `--apply-config` / `--no-apply-config` | `true` | Write openclaw plugin config |
| `--config-backup` / `--no-config-backup` | `true` | Parsed into summary only; **config writer does not create an openclaw.json backup** |
| `--rewrite-manifest` / `--no-rewrite-manifest` | `true` | Rewrite data-dir manifest to `tcvdb` |
| `--fail-if-target-nonempty` / `--no-fail-if-target-nonempty` | `true` | Abort when target already has L0/L1/profiles |
| `--verify-counts` / `--no-verify-counts` | `true` | Post-migrate exact count match (with 10s settle delay) |
| `--bm25-enabled` / `--no-bm25-enabled` | `true` | BM25 encoder for target store + config patch |
| `--bm25-language` | `zh` | `zh` or `en` only |

<Tip>
Back up `openclaw.json` yourself before a live run. Manifest backup is automatic (`manifest.json.migrate.bak`); openclaw config backup is not performed by the writer.
</Tip>

## openclaw.json rewrite

When `--apply-config` is enabled (default), `writeMigrationPluginConfig` merges into:

```text
plugins.entries.<pluginId>.config
```

Default `pluginId` is `memory-tencentdb`. Patch shape:

```json
{
  "storeBackend": "tcvdb",
  "tcvdb": {
    "url": "<tcvdb-url>",
    "username": "<tcvdb-username>",
    "apiKey": "<resolved-key>",
    "database": "<tcvdb-database>",
    "alias": "<tcvdb-alias>",
    "embeddingModel": "<tcvdb-embedding-model>",
    "timeout": 10000
  },
  "bm25": {
    "enabled": true,
    "language": "zh"
  }
}
```

Merge rules:

- Existing entry and nested `tcvdb` / `bm25` objects are shallow-merged (patch wins on overlapping keys).
- Other plugin config keys (embedding, extraction, capture, etc.) are preserved.
- Input is parsed with **JSON5**; output is pretty-printed JSON with a trailing newline.
- Writer requires a single-file config; multi-file layouts are not supported.

Disable automatic rewrite when you manage config out of band:

```bash
npm run migrate-sqlite-to-tcvdb -- \
  ...required flags... \
  --no-apply-config \
  --no-rewrite-manifest \
  --yes
```

## Manifest rewrite

When `--rewrite-manifest` is enabled (default):

| Case | Result |
|---|---|
| No existing manifest | Creates version `1` with `store.type: "tcvdb"` |
| Existing manifest | Copies current file to `.metadata/manifest.json.migrate.bak`, then updates `store` only |

Store info includes `url`, `database`, and optional `alias`. Relative sqlite path is not retained after rewrite.

## Command recipes

### Dry-run preflight

```bash
export TCVDB_API_KEY='***'
npm run migrate-sqlite-to-tcvdb -- \
  --plugin-data-dir ~/.openclaw/memory-tdai \
  --openclaw-config-path ~/.openclaw/openclaw.json \
  --tcvdb-url http://127.0.0.1:80 \
  --tcvdb-username root \
  --tcvdb-api-key-env TCVDB_API_KEY \
  --tcvdb-database agent_memory_prod \
  --tcvdb-embedding-model bge-large-zh \
  --dry-run
```

Stdout is a JSON `MigrationPreflightSummary` with `source.l0Count` / `l1Count` / `profileCount`, `manifestStoreType`, target connection fields, and option flags. No `migration` block is attached on dry-run.

### Full live migration

```bash
cp ~/.openclaw/openclaw.json ~/.openclaw/openclaw.json.bak
npm run migrate-sqlite-to-tcvdb -- \
  --plugin-data-dir ~/.openclaw/memory-tdai \
  --openclaw-config-path ~/.openclaw/openclaw.json \
  --tcvdb-url http://127.0.0.1:80 \
  --tcvdb-username root \
  --tcvdb-api-key-env TCVDB_API_KEY \
  --tcvdb-database agent_memory_prod \
  --tcvdb-embedding-model bge-large-zh \
  --yes
```

On success, summary includes `migration` with migrated counts, target counts, `configWritten`, `manifestWritten`, and optional `manifestBackupPath`.

### L1 only

```bash
# ...required connection flags...
  --layers l1 \
  --yes
```

### L0 + L1 without profiles

```bash
  --layers l0,l1 \
  --yes
```

### Custom SQLite snapshot

```bash
  --sqlite-path /backup/2026-04/vectors-snapshot.db \
  --yes
```

### English BM25

```bash
  --bm25-language en \
  --tcvdb-embedding-model bge-large-en-v1.5 \
  --yes
```

### Dense-only (no BM25 sparse)

```bash
  --no-bm25-enabled \
  --yes
```

### Append into non-empty target (unsafe for strict cutover)

```bash
  --no-fail-if-target-nonempty \
  --no-verify-counts \
  --yes
```

Count verification is usually disabled together with non-empty target, because target totals will not equal source-only counts.

### CI summary artifact

```bash
  --summary-json-path ./migration-report.json \
  --job-id migrate-2026-04-13 \
  --yes
```

## Post-migration checks

After a live run with default flags:

<Steps>
<Step title="Confirm config patch">
Inspect the plugin entry (default id `memory-tencentdb`):

```bash
python3 - <<'PY'
import json, pathlib
p = pathlib.Path.home() / ".openclaw" / "openclaw.json"
cfg = json.loads(p.read_text())
entry = cfg.get("plugins", {}).get("entries", {}).get("memory-tencentdb", {})
c = entry.get("config", entry)
print("storeBackend =", c.get("storeBackend"))
print("tcvdb.database =", (c.get("tcvdb") or {}).get("database"))
print("bm25 =", c.get("bm25"))
assert c.get("storeBackend") == "tcvdb"
assert (c.get("tcvdb") or {}).get("url")
assert (c.get("tcvdb") or {}).get("apiKey")
assert (c.get("tcvdb") or {}).get("database")
print("ok")
PY
```

Runtime factory requires `tcvdb.url`, `tcvdb.apiKey`, and `tcvdb.database` when `storeBackend === "tcvdb"`.
</Step>
<Step title="Confirm manifest store binding">
```bash
cat ~/.openclaw/memory-tdai/.metadata/manifest.json
# expect store.type == "tcvdb"
# optional: .metadata/manifest.json.migrate.bak from pre-migration sqlite binding
```
</Step>
<Step title="Restart Gateway and smoke recall">
```bash
openclaw gateway restart
# Expect store logs with backend=tcvdb (factory tag [memory-tdai][factory])
# Exercise tdai_memory_search / conversation search or a short dialogue
```
</Step>
<Step title="Compare counts from the CLI summary">
Use the printed `migration` object:

- `l1Migrated` / `targetL1Count` vs preflight `source.l1Count`
- `l0Migrated` / `targetL0Count` vs `source.l0Count`
- `profileMigrated` / `targetProfileCount` vs `source.profileCount`

With default `--verify-counts`, a mismatch already failed the process.
</Step>
</Steps>

<Check>
Success signals: process exit 0, stdout JSON with `migration.configWritten: true` (unless disabled), `storeBackend: "tcvdb"` in OpenClaw plugin config, manifest `store.type: "tcvdb"`, Gateway loading without TCVDB init errors.
</Check>

## Failure modes

| Symptom | Likely cause | Recovery |
|---|---|---|
| `Missing required option --...` | Incomplete argv | Pass all required flags; API key via key **or** env |
| `Provide either --tcvdb-api-key or --tcvdb-api-key-env, not both` | Both key inputs set | Keep one form only |
| `Environment variable X is empty or not set` | Bad `--tcvdb-api-key-env` | Export the variable before run |
| `Failed to open sqlite store...` / degraded source | Corrupt or unreadable `vectors.db` | Repair/restore sqlite; re-run dry-run |
| `Target store entered degraded mode...` | TCVDB init/network/auth failure | Fix URL/credentials/network/CA PEM |
| `Target store is not empty (L1=..., L0=..., profiles=...)` | Default nonempty guard | Empty target DB, change `--tcvdb-database`, or `--no-fail-if-target-nonempty` |
| `L1/L0/Profile count verification failed` | Partial write or settle lag | Inspect TCVDB counts; re-run after clean target; avoid disabling verify until root cause is known |
| `Failed to batch migrate L1/L0...` | Batch upsert returned 0 | Check TCVDB write quotas/errors; fix then re-run on empty target |
| `Config migration writer only supports single-file JSON/JSON5` | Unreadable path or non-JSON5 content | Point `--openclaw-config-path` at a valid file |
| `Unsupported layer(s)` / `Unsupported --bm25-language` | Bad enum value | Use `l0,l1,l2,l3` and `zh`/`en` only |
| Bin import fails | Dist not built | Run `npm run build:migrate-sqlite-to-vdb` |

Exit behavior: uncaught errors set `process.exitCode = 1` and print `[memory-tdai][migrate-cli] <message>` on stderr.

## Operational notes

- **Idempotency:** Default path is a **cutover** into an empty TCVDB database, not a merge. Upserts may overwrite by id if you force nonempty targets, but count verification will fail unless disabled.
- **Secrets:** Prefer `--tcvdb-api-key-env`. Live config rewrite writes `apiKey` into `openclaw.json` in cleartext — protect file permissions and backups.
- **Data dir:** SQLite file and local JSONL trees remain after migration; runtime switches via `storeBackend`, not by deleting local files.
- **Plugin id vs log tag:** Config key defaults to `memory-tencentdb`; many runtime log prefixes still say `memory-tdai`.
- **Does not depend on `openclaw/plugin-sdk`:** migration modules import core store code and `json5` only.

## Related pages

<CardGroup>
<Card title="Configure storage backends" href="/configure-storage">
Choose sqlite vs tcvdb, required tcvdb fields, embedding, and BM25 language.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
Full plugin schema for `storeBackend`, `tcvdb`, `bm25`, and related keys.
</Card>
<Card title="CLI reference" href="/cli-reference">
Package bins including `migrate-sqlite-to-tcvdb` and build scripts.
</Card>
<Card title="Data directories" href="/data-directories">
On-disk layout for `~/.openclaw/memory-tdai`, vectors.db, persona, and scene_blocks.
</Card>
<Card title="Migrate from memory-tdai package" href="/package-rename">
Package rename workflow (not storage backend migration).
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
Runtime failure modes after store switches (embedding, recall, gateway).
</Card>
</CardGroup>