# Configure storage backends

> Choose sqlite (sqlite-vec + FTS5) or tcvdb, required tcvdb fields, embedding provider quadruplets, BM25 language, and verification of store health.

- Repository: TencentCloud/TencentDB-Agent-Memory
- GitHub: https://github.com/TencentCloud/TencentDB-Agent-Memory
- Human docs: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97
- Complete Markdown: https://grok-wiki.com/public/docs/tencentcloud-tencentdb-agent-memory-14fefdd76c97/llms-full.txt

## Source Files

- `src/core/store/factory.ts`
- `src/core/store/sqlite.ts`
- `src/core/store/tcvdb.ts`
- `src/core/store/embedding.ts`
- `src/config.ts`
- `openclaw.plugin.json`

---

---
title: "Configure storage backends"
description: "Choose sqlite (sqlite-vec + FTS5) or tcvdb, required tcvdb fields, embedding provider quadruplets, BM25 language, and verification of store health."
---

`storeBackend` selects the runtime memory index: **`sqlite`** (default local `vectors.db` with sqlite-vec + FTS5) or **`tcvdb`** (Tencent Cloud VectorDB with server-side dense embedding). `createStoreBundle()` in the store factory builds an `IMemoryStore`, an embedding service (client-side for sqlite, `NoopEmbeddingService` for tcvdb), and an optional BM25 encoder, then `init()` marks the store healthy or degraded before recall, tools, and L1 dedup use it.

## Backend choice

| `storeBackend` | Default | On-disk / remote surface | Dense vectors | Keyword / sparse | Hybrid |
| --- | --- | --- | --- | --- | --- |
| `sqlite` | yes | `{dataDir}/vectors.db` | Client embedding + sqlite-vec `vec0` | FTS5 (`l1_fts`, `l0_fts`) when available | Client RRF (not native hybrid) |
| `tcvdb` | no | TCVDB database + collections | Server-side collection embedding | Local BM25 sparse via `@tencentdb-agent-memory/tcvdb-text` | Native `hybridSearch` when BM25 is enabled |

```mermaid
flowchart TB
  subgraph cfg ["Plugin config"]
    SB["storeBackend"]
    EMB["embedding.*"]
    TCV["tcvdb.*"]
    BM["bm25.*"]
  end

  subgraph factory ["createStoreBundle"]
    SB --> SW{"sqlite | tcvdb"}
    SW -->|sqlite| VS["VectorStore<br/>vectors.db"]
    SW -->|tcvdb| TS["TcvdbMemoryStore"]
    EMB --> ES["OpenAI / ZeroEntropy / none"]
    BM --> BE["BM25LocalEncoder or undefined"]
    TCV --> TS
    ES --> VS
    BE --> TS
    BE --> VS
    TS --> NOOP["NoopEmbeddingService"]
  end

  subgraph init ["pipeline-factory _doInitStores"]
    VS --> INIT["store.init(providerInfo)"]
    TS --> INIT
    INIT --> OK{"isDegraded()?"}
    OK -->|yes| DEG["vectorStore = undefined<br/>keyword / no-vector path"]
    OK -->|no| LIVE["vector + FTS / hybrid ready"]
  end
```

Unknown `storeBackend` values parse as **`sqlite`**. Zero-config (`{}`) uses sqlite with `embedding.provider: "none"` (keyword/FTS path only until you configure a remote embedding quadruplet).

## SQLite backend

### Config

```json
{
  "storeBackend": "sqlite",
  "embedding": {
    "enabled": true,
    "provider": "openai",
    "baseUrl": "https://api.openai.com/v1",
    "apiKey": "<KEY>",
    "model": "text-embedding-3-small",
    "dimensions": 1536
  },
  "bm25": {
    "enabled": true,
    "language": "zh"
  }
}
```

OpenClaw plugin placement (plugin id `memory-tencentdb`, command alias `memory-tdai`):

```json
{
  "plugins": {
    "entries": {
      "memory-tencentdb": {
        "enabled": true,
        "config": {
          "storeBackend": "sqlite"
        }
      }
    }
  }
}
```

### What gets created

| Artifact | Path / name | Role |
| --- | --- | --- |
| SQLite file | `{pluginDataDir}/vectors.db` | Single DB for L0/L1 metadata, vec0, FTS5 |
| L1 metadata | `l1_records` | Structured memories |
| L1 vectors | `l1_vec` (`vec0`, cosine) | Only when `embedding.dimensions > 0` |
| L0 metadata | `l0_conversations` | Raw messages |
| L0 vectors | `l0_vec` | Same dimension rule as L1 |
| FTS5 | `l1_fts`, `l0_fts` | Best-effort; jieba-segmented index text + original UNINDEXED display text |
| Embedding meta | `embedding_meta` | Provider/model/dimensions change detection |

Constructor pragmas: `busy_timeout=5000`, WAL, `cache_size=-65536`, `mmap_size=134217728`, `wal_autocheckpoint=1000`.

### Embedding dimensions and deferred vec0

| Condition | Behavior |
| --- | --- |
| `embedding.provider = "none"` (default) | `enabled` forced false; `dimensions` defaults to `0`; **vec0 tables are not created** |
| Remote provider fully configured | Client `EmbeddingService` embeds on write/search; vec0 tables use configured dimensions |
| Provider/model/dimensions change | `init()` drops vector tables, returns `needsReindex: true` for re-embed |

Factory only constructs a client embedding service when **all** of: `embedding.enabled`, `provider !== "local"`, and non-empty `apiKey`.

### Capabilities (`getCapabilities()`)

| Flag | Value |
| --- | --- |
| `vectorSearch` | `true` only when vec0 tables are ready (`dimensions > 0` and init succeeded) |
| `ftsSearch` | `true` when FTS5 tables initialized |
| `nativeHybridSearch` | always `false` |
| `sparseVectors` | always `false` |

FTS5 creation is best-effort: if unavailable, keyword search degrades; the store does not enter full degraded mode solely for missing FTS5. **sqlite-vec load failure** or schema init failure sets `isDegraded() === true` and all write/search ops become no-ops.

## Tencent VectorDB backend

### Required fields

`createStoreBundle` **throws** (caught by store init → full store unavailable) unless:

| Field | Required | Default | Notes |
| --- | --- | --- | --- |
| `tcvdb.url` | **yes** | `""` | Instance URL, e.g. `http://10.0.1.1:8100` |
| `tcvdb.apiKey` | **yes** | `""` | API key |
| `tcvdb.database` | **yes** | `""` | Must be set explicitly; unique name per instance |
| `tcvdb.username` | no | `"root"` | Account name |
| `tcvdb.embeddingModel` | no | `"bge-large-zh"` | Server-side collection embedding model |
| `tcvdb.timeout` | no | `10000` | Request timeout (ms) |
| `tcvdb.alias` | no | `""` | Optional label in `manifest.json` / store snapshot |
| `tcvdb.caPemPath` | no | — | CA PEM path for HTTPS |

```json
{
  "storeBackend": "tcvdb",
  "tcvdb": {
    "url": "http://10.0.1.1:8100",
    "username": "root",
    "apiKey": "<TCVDB_API_KEY>",
    "database": "my_agent_memory",
    "embeddingModel": "bge-large-zh",
    "timeout": 10000
  },
  "bm25": {
    "enabled": true,
    "language": "zh"
  }
}
```

### Collections and embedding

On `init()`, the store:

1. `createDatabase()` (idempotent; if just created, waits ~5s before collections).
2. Creates collections with database-prefixed names:
   - `{database}_l1_memories` — L1 text → vector via collection embedding on field `text`
   - `{database}_l0_conversations` — L0 `message_text` → vector
   - `{database}_profiles` — L2/L3 profiles (embedding disabled; placeholder 1-D vector)
3. Prefers **DISK_FLAT** vector index (`dimension: 1024`, COSINE); falls back to **HNSW** on API code `15113` / unsupported DISK_FLAT messages.
4. Always indexes `sparse_vector` (inverted, IP) for hybrid/BM25 paths.

Client-side embedding is **not** used: the factory attaches `NoopEmbeddingService` (`getDimensions() === 0`, empty float arrays). Do not expect sqlite-style `embedding.*` to drive TCVDB dense vectors.

### Capabilities

| Flag | Value |
| --- | --- |
| `vectorSearch` | `true` (when not degraded) |
| `ftsSearch` | `true` if BM25 encoder present (sparse path stands in for pure FTS) |
| `nativeHybridSearch` | `true` if BM25 encoder present |
| `sparseVectors` | `true` if BM25 encoder present |

Without BM25, dense-only search remains; hybrid/sparse paths drop out.

## Embedding provider quadruplet (sqlite)

Remote embedding for **sqlite** needs a consistent quadruplet (plus optional knobs). `parseConfig` validates and **disables embedding without throwing** when fields are missing, storing `embedding.configError`.

### Required remote fields

| Field | Role |
| --- | --- |
| `embedding.provider` | Any non-`none` / non-`local` label (e.g. `openai`, `deepseek`, `azure`); `zeroentropy` uses a native wire format |
| `embedding.baseUrl` | API base URL |
| `embedding.apiKey` | Credential |
| `embedding.model` | Model id |
| `embedding.dimensions` | Positive integer matching the model |

### Provider modes

| `provider` | Result |
| --- | --- |
| `"none"` or omitted | Embedding off; dimensions `0`; vec0 deferred |
| `"local"` | Treated as disabled for user config; `configError` set; not wired from public schema |
| `"qclaw"` | Requires **`proxyUrl` + baseUrl + apiKey + model + dimensions**; requests go through the proxy with `Remote-URL` header |
| `"zeroentropy"` | `ZeroEntropyEmbeddingService` (`/v1/models/embed`); still needs the same quadruplet |
| Any other string with full quadruplet | `OpenAIEmbeddingService` (OpenAI-compatible `/embeddings`) |

### Optional knobs

| Field | Default | Purpose |
| --- | --- | --- |
| `enabled` | `true` (schema); forced `false` when provider is `none` or config invalid | Master switch |
| `sendDimensions` | `true` | Include `dimensions` in request body; set `false` for fixed-dim models (e.g. BGE-M3) that reject Matryoshka params |
| `maxInputChars` | `5000` | Truncate oversize inputs with a warning |
| `timeoutMs` | `10000` | Per-call timeout (retries up to 3 in service) |
| `recallTimeoutMs` | — | Overrides timeout on user-facing recall path |
| `captureTimeoutMs` | — | Overrides timeout on background capture/dedup path |
| `conflictRecallTopK` | `5` | Top-K for L1 conflict/dedup recall |

### Examples

<Tabs>
  <Tab title="OpenAI-compatible">
```json
{
  "storeBackend": "sqlite",
  "embedding": {
    "provider": "openai",
    "baseUrl": "https://api.openai.com/v1",
    "apiKey": "<KEY>",
    "model": "text-embedding-3-small",
    "dimensions": 1536,
    "sendDimensions": true
  }
}
```
  </Tab>
  <Tab title="Fixed-dim self-hosted (BGE-M3)">
```json
{
  "embedding": {
    "provider": "openai",
    "baseUrl": "http://your-host:port/v1",
    "apiKey": "<KEY>",
    "model": "bge-m3",
    "dimensions": 1024,
    "sendDimensions": false
  }
}
```
  </Tab>
  <Tab title="qclaw proxy">
```json
{
  "embedding": {
    "provider": "qclaw",
    "proxyUrl": "http://127.0.0.1:PORT",
    "baseUrl": "https://upstream.example/v1",
    "apiKey": "<KEY>",
    "model": "your-model",
    "dimensions": 1024
  }
}
```
  </Tab>
  <Tab title="ZeroEntropy">
```json
{
  "embedding": {
    "provider": "zeroentropy",
    "baseUrl": "https://api.zeroentropy.dev",
    "apiKey": "<KEY>",
    "model": "zembed-1",
    "dimensions": 1536
  }
}
```
  </Tab>
</Tabs>

<Warning>
Incomplete remote embedding config disables vector search for sqlite; the plugin keeps running. FTS5/keyword paths remain if available. Missing tcvdb credentials fail store creation instead of soft-disabling embedding.
</Warning>

## BM25 language

`bm25` configures the **local** sparse encoder (`BM25LocalEncoder` → `BM25Encoder.default(language)` from `@tencentdb-agent-memory/tcvdb-text`).

| Field | Type | Default | Values |
| --- | --- | --- | --- |
| `bm25.enabled` | boolean | `true` | When `false`, factory omits the encoder |
| `bm25.language` | string | `"zh"` | `"zh"` or `"en"` only; any other value becomes `"zh"` |

```json
{
  "bm25": {
    "enabled": true,
    "language": "en"
  }
}
```

| Backend | BM25 role |
| --- | --- |
| **tcvdb** | Encode documents on upsert (`encodeTexts`) and queries on search (`encodeQueries`); enables sparse + hybrid + `ftsSearch`/`nativeHybridSearch` capability flags |
| **sqlite** | Encoder is still constructed when enabled, but sqlite keyword path uses **FTS5 + jieba**, not sparse vectors (`sparseVectors: false`) |

Encode failures log a warning and return empty sparse vectors (dense-only search continues).

## Store lifecycle and health

### Init path

1. `createStoreBundle(cfg, { dataDir, logger })`
2. `vectorStore.init(embeddingService?.getProviderInfo())`
3. If `isDegraded()` → store and embedding cleared; pipeline falls back (keyword dedup / no vector recall)
4. On success, write or diff `<dataDir>/.metadata/manifest.json` store binding (first-write only; later mismatches log at debug)

### Degraded and failure signals

| Signal | Meaning |
| --- | --- |
| Log `Store created: backend=sqlite, dbPath=..., dimensions=...` | Factory succeeded |
| Log `Store created: backend=tcvdb, database=..., model=...` | Factory succeeded |
| Log `Store initialized: backend=..., provider=...` | `init()` healthy |
| Log `Store is in degraded mode, falling back to keyword dedup` | `isDegraded()` after init |
| Log `Store init failed; vector/FTS recall and dedup...` | Thrown factory error (e.g. missing tcvdb fields) or init exception |
| Log `Failed to load sqlite-vec extension` | Full sqlite store degraded |
| Log `FTS5 tables NOT available` | Keyword FTS only missing; store may still be healthy for vectors |
| `embedding.configError` | Remote embedding quadruplet incomplete; embedding disabled |

### Gateway health (Hermes / standalone sidecar)

`GET /health` (no auth):

```json
{
  "status": "ok",
  "version": "<package version>",
  "uptime": 123,
  "stores": {
    "vectorStore": true,
    "embeddingService": true
  }
}
```

| Field | Semantics |
| --- | --- |
| `status` | `"ok"` if core has a vector store instance; else `"degraded"` |
| `stores.vectorStore` | Whether `getVectorStore()` is set |
| `stores.embeddingService` | Whether `getEmbeddingService()` is set (for tcvdb this is the noop service when store init succeeded) |

Operators can also use `memory-tencentdb-ctl health` / gateway control paths documented under Gateway control.

### Manifest store binding

After first successful init:

```json
{
  "version": 1,
  "createdAt": "2026-04-01T22:00:00.000Z",
  "store": {
    "type": "sqlite",
    "sqlite": { "path": "vectors.db" }
  },
  "seed": null
}
```

TCVDB shape:

```json
{
  "store": {
    "type": "tcvdb",
    "tcvdb": {
      "url": "http://10.0.1.1:8100",
      "database": "my_agent_memory",
      "alias": "prod"
    }
  }
}
```

Intentional backend switches log store-binding diffs; they do not auto-migrate data. Use the SQLite→TCVDB migration tool for data movement.

### Practical verification checklist

<Steps>
  <Step title="Confirm backend selection">
    Set `storeBackend` to `sqlite` or `tcvdb`. For tcvdb, set `url`, `apiKey`, and a unique `database` before restarting the gateway/plugin.
  </Step>
  <Step title="Configure embedding (sqlite only)">
    Provide `provider`, `baseUrl`, `apiKey`, `model`, and `dimensions`. Set `sendDimensions: false` for fixed-dimension OSS models.
  </Step>
  <Step title="Set BM25 language">
    Match corpus language: `bm25.language: "zh"` or `"en"`. Disable only if you intentionally want dense-only tcvdb behavior.
  </Step>
  <Step title="Restart and read logs">
    Expect `[memory-tdai][factory] Store created` and `Store initialized` without degraded/init-failed warnings.
  </Step>
  <Step title="Probe health and data">
    Gateway: `GET /health` → `status: "ok"` and `stores.vectorStore: true`. OpenClaw sqlite: confirm `~/.openclaw/memory-tdai/vectors.db` (or host data dir) and `.metadata/manifest.json`. Exercise `tdai_memory_search` / `tdai_conversation_search` with hybrid strategy.
  </Step>
</Steps>

## Common misconfigurations

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| No vector recall on sqlite | `provider: "none"` or incomplete quadruplet | Fill embedding quadruplet; restart |
| HTTP 400 on embed (matryoshka) | `sendDimensions: true` on fixed-dim model | Set `sendDimensions: false` |
| Store init failed / no vectorStore | tcvdb missing `url`/`apiKey`/`database` | Set required `tcvdb.*` fields |
| Hybrid weak on tcvdb | `bm25.enabled: false` or encode errors | Enable BM25; set correct `language` |
| Full degraded sqlite | sqlite-vec native load failed | Node ≥ 22.16 with sqlite-vec install; check extension load errors |
| Keyword-only after switch | Reindex needed after model/dim change | Allow reindex path after `needsReindex`; verify dimensions match model |
| qclaw always disabled | Missing `proxyUrl` or quadruplet field | Add all five: proxyUrl + baseUrl + apiKey + model + dimensions |

## Related pages

<CardGroup>
  <Card title="Configuration reference" href="/configuration-reference">
    Full schema for storeBackend, embedding, tcvdb, bm25, recall, and degrade rules.
  </Card>
  <Card title="Migrate SQLite to Tencent VectorDB" href="/migrate-sqlite-tcvdb">
    migrate-sqlite-to-tcvdb dry-run, layer selection, and openclaw.json rewrite.
  </Card>
  <Card title="Data directories" href="/data-directories">
    vectors.db, conversations, records, scene_blocks, and Hermes MEMORY_TENCENTDB_ROOT trees.
  </Card>
  <Card title="Agent tools" href="/agent-tools">
    tdai_memory_search / tdai_conversation_search strategies and RRF hybrid merge.
  </Card>
  <Card title="Gateway HTTP API" href="/gateway-http-api">
    GET /health and store-backed recall/search endpoints.
  </Card>
  <Card title="Troubleshooting" href="/troubleshooting">
    Embedding degrade, silent plugin, and recovery probes.
  </Card>
</CardGroup>