# RAG pipeline

> Complete vector retrieval workflow from `examples/rag_integration.py`: load all bundle markdown, split on headers, embed into Chroma, query with `retriever.invoke`, and swap embedding providers without changing OKF storage.

- Repository: EliaszDev/hermes-okf
- GitHub: https://github.com/EliaszDev/hermes-okf
- Human docs: https://grok-wiki.com/public/docs/eliaszdev-hermes-okf-b71befaafe02
- Complete Markdown: https://grok-wiki.com/public/docs/eliaszdev-hermes-okf-b71befaafe02/llms-full.txt

## Source Files

- `examples/rag_integration.py`
- `pyproject.toml`
- `src/hermes_okf/bundle.py`
- `README.md`
- `src/hermes_okf/search.py`

---

---
title: "RAG pipeline"
description: "Complete vector retrieval workflow from `examples/rag_integration.py`: load all bundle markdown, split on headers, embed into Chroma, query with `retriever.invoke`, and swap embedding providers without changing OKF storage."
---

Vector retrieval over an OKF bundle is an optional sidecar: `hermes-okf` core persists concepts as filesystem markdown with only `pyyaml`, while the `[rag]` extra wires LangChain loaders, `MarkdownHeaderTextSplitter`, Chroma persistence, and `retriever.invoke` against that same bundle. OKF files are never rewritten during indexing — embeddings live in a separate Chroma directory.

<Info>
RAG is not exposed through `hermes-okf` or `hermes okf` CLI commands. Use `examples/rag_integration.py` for a standalone script, or `HermesOKFProvider.rag_search()` from Python.
</Info>

## Architecture

The pipeline reads from the OKF bundle root, chunks by markdown headers, embeds into Chroma, and queries through a LangChain retriever. Full-text search via `SearchIndex` remains the default in-process path; RAG adds semantic similarity on top without replacing OKF storage.

```mermaid
flowchart LR
  subgraph okf["OKF storage (unchanged)"]
    B["OKFBundle.root"]
    MD["**/*.md concepts"]
    B --> MD
  end

  subgraph rag["RAG sidecar (optional [rag] extra)"]
    DL["DirectoryLoader + TextLoader"]
    SP["MarkdownHeaderTextSplitter"]
    CH["Chroma.from_documents"]
    RT["retriever.invoke"]
    DL --> SP --> CH --> RT
  end

  subgraph embed["Embedding provider (swappable)"]
    EM["OpenAIEmbeddings or compatible"]
  end

  MD --> DL
  EM --> CH
  CH --> VS["persist_directory"]
  VS --> RT
```

| Layer | Module / artifact | Role |
|-------|-------------------|------|
| Source of truth | `OKFBundle` | Filesystem bundle at `bundle.root`; concepts are `.md` + YAML frontmatter |
| Load | `DirectoryLoader`, `TextLoader` | Recursively loads `**/*.md` under the bundle root with UTF-8 encoding |
| Split | `MarkdownHeaderTextSplitter` | Splits on `#` and `##` headers into LangChain `Document` chunks |
| Index | `Chroma.from_documents` | Embeds chunks and persists vectors to disk |
| Query | `vectorstore.as_retriever` | Returns top-`k` chunks via `retriever.invoke(query)` |
| Built-in alternative | `SearchIndex` | Stdlib inverted-index full-text search; no embeddings required |

<Warning>
`DirectoryLoader` includes every `**/*.md` file under the bundle root — including reserved `index.md` and `log.md`. `OKFBundle.list_concepts()` skips those files, but the RAG loader does not.
</Warning>

## Prerequisites

<Steps>
<Step title="Install the RAG extra">

```bash
pip install hermes-okf[rag]
```

This pulls in `langchain`, `langchain-community`, `langchain-chroma`, and `langchain-openai` (see `pyproject.toml` optional-dependencies).

</Step>

<Step title="Prepare an OKF bundle">

Initialize or point at an existing bundle. The example uses `./my_knowledge`; the Hermes provider defaults to `~/.hermes/okf_memory`.

```bash
hermes-okf init ./my_knowledge
```

</Step>

<Step title="Configure an embedding provider">

`OpenAIEmbeddings` (default in examples) expects credentials for your chosen provider — typically `OPENAI_API_KEY` for OpenAI-compatible endpoints. Swap the embedding class without touching OKF files.

</Step>
</Steps>

## Pipeline stages

### 1. Bind the bundle

`OKFBundle` resolves the bundle root path. Pass `str(bundle.root)` to LangChain loaders.

```python
from hermes_okf.bundle import OKFBundle

bundle = OKFBundle("./my_knowledge")
```

### 2. Load all markdown

`DirectoryLoader` walks the bundle tree and loads every `.md` file as a LangChain document.

```python
from langchain_community.document_loaders import DirectoryLoader, TextLoader

loader = DirectoryLoader(
    str(bundle.root),
    glob="**/*.md",
    loader_cls=TextLoader,
    loader_kwargs={"encoding": "utf-8"},
)
docs = loader.load()
```

Each loaded document carries a `source` metadata field (file path) that flows into Chroma and query results.

### 3. Split on headers

`MarkdownHeaderTextSplitter` breaks documents at `#` (Header 1) and `##` (Header 2) boundaries. Header text is attached to chunk metadata.

```python
from langchain.text_splitter import MarkdownHeaderTextSplitter

splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on=[("#", "Header 1"), ("##", "Header 2")]
)
splits = []
for doc in docs:
    splits.extend(splitter.split_text(doc.page_content))
```

### 4. Embed into Chroma

`Chroma.from_documents` embeds all splits and writes a persistent vector store. The example uses a project-local directory; the provider writes under the bundle.

```python
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

vectorstore = Chroma.from_documents(
    documents=splits,
    embedding=OpenAIEmbeddings(),
    persist_directory="./chroma_okf_db",
)
```

### 5. Query with the retriever

Build a retriever with `search_kwargs={"k": 5}` and invoke it with a natural-language query string.

```python
retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
results = retriever.invoke("What GPU decisions did we make?")

for r in results:
    print(f"---\n{r.page_content[:300]}...\n")
```

<ResponseField name="Document" type="langchain Document">
Each result is a LangChain `Document` with `page_content` (chunk text) and `metadata` (including `source` file path and header keys from the splitter).
</ResponseField>

## Complete example script

The canonical reference is `examples/rag_integration.py`. Run it after installing `[rag]` and populating a bundle:

```bash
pip install hermes-okf[rag]
python examples/rag_integration.py
```

The script chains all five stages: `OKFBundle` → `DirectoryLoader` → `MarkdownHeaderTextSplitter` → `Chroma.from_documents` → `retriever.invoke`.

## Provider integration

`HermesOKFProvider` in `hermes_okf.hermes_integration` embeds the same pipeline in `rag_search()` and `_build_rag_index()`. Differences from the standalone example:

| Aspect | `examples/rag_integration.py` | `HermesOKFProvider` |
|--------|------------------------------|---------------------|
| Bundle path | `OKFBundle("./my_knowledge")` | `self.config.bundle_path` (default `~/.hermes/okf_memory`) |
| Chroma persist dir | `./chroma_okf_db` | `{bundle_path}/.chroma` |
| Embedding model | `OpenAIEmbeddings()` (default) | `OpenAIEmbeddings(model=self.config.rag_model)` |
| Index build | Eager in script | Lazy — `_build_rag_index()` runs when `.chroma` is missing |
| Return shape | Raw `Document` list | `list[dict]` with `source` and `content` (truncated to 500 chars) |

```python
from hermes_okf import get_provider

provider = get_provider()
results = provider.rag_search("deployment strategies for Python services", top_k=5)

for r in results:
    print(f"{r['source']}: {r['content'][:100]}")
```

<Note>
`rag_search()` raises `ImportError` with install instructions if `hermes-okf[rag]` is not installed. It does not check `enable_rag` before running — call the method explicitly when you want semantic search.
</Note>

## Swap embedding providers

OKF storage is embedding-agnostic. Only the LangChain `embedding` / `embedding_function` argument changes. The example comment notes `OpenRouterEmbeddings` as an alternative to `OpenAIEmbeddings`.

<Tabs>
<Tab title="OpenAI (default)">

```python
from langchain_openai import OpenAIEmbeddings

embedding = OpenAIEmbeddings()
# Provider default model: openai/text-embedding-3-small
embedding = OpenAIEmbeddings(model="openai/text-embedding-3-small")
```

</Tab>
<Tab title="OpenRouter or other compatible">

```python
# Any LangChain Embeddings implementation works here.
# from langchain_openai import OpenAIEmbeddings  # OpenRouter-compatible base URL
# embedding = OpenAIEmbeddings(base_url="https://openrouter.ai/api/v1", ...)

vectorstore = Chroma.from_documents(
    documents=splits,
    embedding=your_embedding_instance,
    persist_directory="./chroma_okf_db",
)
```

</Tab>
</Tabs>

Rebuilding the Chroma index after swapping models is required — delete the persist directory and re-run `from_documents` or `_build_rag_index()`.

## Configuration

RAG-related fields on `HermesOKFConfig`:

<ParamField body="enable_rag" type="boolean" default="false">
Loaded from `HERMES_OKF_ENABLE_RAG` (values `1`, `true`, `yes`), `~/.hermes/hermes-okf.yaml`, or `plugins.hermes_okf` in `~/.hermes/config.yaml`. Stored on the config object; `rag_search()` does not gate on this flag today.
</ParamField>

<ParamField body="rag_model" type="string" default="openai/text-embedding-3-small">
Embedding model passed to `OpenAIEmbeddings(model=...)` in `_build_rag_index()` and `rag_search()`.
</ParamField>

<ParamField body="bundle_path" type="string" default="~/.hermes/okf_memory">
OKF bundle root. Chroma index is written to `{bundle_path}/.chroma` by the provider.
</ParamField>

Environment variable resolution for `enable_rag`:

```bash
export HERMES_OKF_ENABLE_RAG=true
```

## RAG vs full-text search

| Capability | `SearchIndex` | RAG (Chroma) |
|------------|---------------|--------------|
| Dependency | Core (`pyyaml` only) | `hermes-okf[rag]` extra |
| Match style | Token overlap / optional `rapidfuzz` | Vector semantic similarity |
| Index location | In-memory, rebuilt per session | Persistent Chroma directory |
| CLI exposure | `hermes-okf search`, `hermes okf search` | Python API only |
| OKF writes | Reads concepts via `list_concepts()` | Reads all `**/*.md` via `DirectoryLoader` |

Use full-text search for exact keyword recall; use RAG when queries need semantic similarity ("find things like this idea") over bundle content.

## Persistence and reindexing

- **Standalone example:** vectors persist in `./chroma_okf_db` (or any path you pass to `persist_directory`).
- **Provider:** vectors persist in `{bundle_path}/.chroma`.
- **Stale index:** the provider rebuilds only when `.chroma` is absent. After OKF concept writes, delete the Chroma directory and call `_build_rag_index()` or re-run the example script to refresh embeddings.
- **OKF unchanged:** concept CRUD through `OKFBundle.write_concept()` and agent memory hooks continue writing plain markdown; RAG is a read-side projection.

## Failure modes

<AccordionGroup>
<Accordion title="ImportError: RAG requires hermes-okf[rag]">

Install the optional extra:

```bash
pip install hermes-okf[rag]
```

</Accordion>

<Accordion title="Empty or irrelevant query results">

Confirm the bundle contains markdown with meaningful `#` / `##` structure. Chunks are header-bound — flat files with no headers become single large chunks. Verify the Chroma index was built after the latest bundle writes.

</Accordion>

<Accordion title="Embedding API errors">

Check provider credentials and network access for your chosen embedding class. `OpenAIEmbeddings` requires a valid API key for the configured endpoint.

</Accordion>

<Accordion title="Reserved files indexed unexpectedly">

`DirectoryLoader` does not exclude `index.md` or `log.md`. Filter documents after `loader.load()` if those files should not appear in semantic results.

</Accordion>
</AccordionGroup>

## Next

<CardGroup>
<Card title="Enable RAG" href="/enable-rag">
Install `[rag]`, config flags, and provider-neutral embedding selection.
</Card>
<Card title="OKF bundle model" href="/okf-bundle-model">
Filesystem layout, concept frontmatter, and reserved files that feed the loader.
</Card>
<Card title="Python SDK reference" href="/python-sdk-reference">
`OKFBundle`, `SearchIndex`, and `get_provider()` public API.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
`HermesOKFConfig` fields, env vars, and resolution order.
</Card>
<Card title="Hermes provider integration" href="/hermes-provider-integration">
Session hooks, `search()`, and `rag_search()` on `HermesOKFProvider`.
</Card>
</CardGroup>