# Local MCP Servers: Email, Images, Memory, RAG as First-Class Tools

> The mcp_servers/ tree ships Odysseus-native Model Context Protocol servers so the agent can call email, image generation, memory, and RAG through the same MCP interface used for third-party tools — wired through mcp_manager and builtin_mcp.

- Repository: pewdiepie-archdaemon/odysseus
- GitHub: https://github.com/pewdiepie-archdaemon/odysseus
- Human wiki: https://grok-wiki.com/public/wiki/pewdiepie-archdaemon-odysseus-8b8805c93124
- Complete Markdown: https://grok-wiki.com/public/wiki/pewdiepie-archdaemon-odysseus-8b8805c93124/llms-full.txt

## Source Files

- `mcp_servers/email_server.py`
- `mcp_servers/image_gen_server.py`
- `mcp_servers/memory_server.py`
- `mcp_servers/rag_server.py`
- `src/mcp_manager.py`
- `src/builtin_mcp.py`
- `routes/mcp_routes.py`

---

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:
- [mcp_servers/email_server.py](mcp_servers/email_server.py)
- [mcp_servers/image_gen_server.py](mcp_servers/image_gen_server.py)
- [mcp_servers/memory_server.py](mcp_servers/memory_server.py)
- [mcp_servers/rag_server.py](mcp_servers/rag_server.py)
- [mcp_servers/_common.py](mcp_servers/_common.py)
- [src/mcp_manager.py](src/mcp_manager.py)
- [src/builtin_mcp.py](src/builtin_mcp.py)
- [routes/mcp_routes.py](routes/mcp_routes.py)
</details>

# Local MCP Servers: Email, Images, Memory, RAG as First-Class Tools

Most agent frameworks treat the Model Context Protocol (MCP) as a way to consume *other people's* tool servers — a Slack adapter here, a Notion server there. Odysseus inverts that: its own core capabilities — IMAP/SMTP, image generation, the memory store, and the RAG index — are shipped as four Python MCP servers under `mcp_servers/`, started as stdio subprocesses on boot, and reachable through the exact same `McpManager` plumbing that handles a third-party Playwright server. The agent does not see a "built-in tool" path and an "external tool" path; it sees one MCP tool surface where some names happen to start with `mcp__email__` or `mcp__rag__`.

This page covers how those servers are structured, how they are wired in through `builtin_mcp.py` and `mcp_manager.py`, and the design choices a builder should notice — including which servers stayed as MCP processes and which ones were deliberately pulled back into the host.

## What "first-class" actually means here

A built-in MCP server in Odysseus has the same contract as a third-party one. Each script in `mcp_servers/` instantiates `mcp.server.Server`, declares tools via `@server.list_tools()`, handles invocations via `@server.call_tool()`, and runs over `stdio_server()`. For example, `image_gen_server.py` is essentially:

```python
# mcp_servers/image_gen_server.py
server = Server("image_gen")

@server.list_tools()
async def list_tools() -> list[Tool]:
    return [Tool(name="generate_image", description=..., inputSchema=...)]

@server.call_tool()
async def call_tool(name, arguments) -> list[TextContent]:
    ...

async def run():
    async with stdio_server() as (read_stream, write_stream):
        await server.run(read_stream, write_stream, server.create_initialization_options())
```

That is the exact MCP server shape an external vendor would write. The only Odysseus-specific bits are an `sys.path.insert(0, ...)` so the subprocess can import the host codebase, and a small `_common.py` of shared output limits and timeouts.

Sources: [mcp_servers/image_gen_server.py:13-39](mcp_servers/image_gen_server.py), [mcp_servers/image_gen_server.py:160-166](mcp_servers/image_gen_server.py), [mcp_servers/_common.py:1-19](mcp_servers/_common.py)

## The four servers and what each owns

| Server (id) | Script | Tools exposed | What it actually does |
|---|---|---|---|
| `email` | `email_server.py` | `list_email_accounts`, `list_emails`, `read_email`, `search_emails`, `send_email`, `reply_to_email`, `archive_email`, `delete_email`, `mark_email_read`, `bulk_email`, `download_attachment` | Multi-account IMAP/SMTP using `imaplib`/`smtplib`, reads accounts from `data/app.db :: email_accounts`, decrypts SMTP/IMAP passwords via `src.secret_storage`, pulls AI-summary text from `email_cache.db` |
| `image_gen` | `image_gen_server.py` | `generate_image` | Calls an OpenAI-compatible `/images/generations` endpoint (`gpt-image-1.5`, `gpt-image-1`, `dall-e-3` auto-detect), saves PNGs to `data/generated_images/`, records each in the `GalleryImage` table |
| `memory` | `memory_server.py` | `manage_memory` (list/add/edit/delete/search) | Wraps the host `MemoryManager`, optionally mirrors entries into a `MemoryVectorStore` for semantic search |
| `rag` | `rag_server.py` | `manage_rag` (list/add_directory/remove_directory) | Drives the RAG singleton and `PersonalDocsManager` — indexing and removing user document directories |

Each tool keeps its schema deliberately small. The email server, for instance, packs eleven distinct operations into one server but exposes a shared `ACCOUNT_PROP` so every call can target a specific mailbox by name, address, or id; the bulk operation accepts either an explicit `uids` list or `all_unread: true` to avoid one-RPC-per-message storms.

Sources: [mcp_servers/email_server.py:1036-1228](mcp_servers/email_server.py), [mcp_servers/memory_server.py:47-72](mcp_servers/memory_server.py), [mcp_servers/rag_server.py:46-65](mcp_servers/rag_server.py), [mcp_servers/image_gen_server.py:22-39](mcp_servers/image_gen_server.py)

## How they get wired in: `builtin_mcp.py` and `McpManager`

On startup, `register_builtin_servers` walks a static `_BUILTIN_SERVERS` dict and calls `mcp_manager.connect_server(...)` for each one, using the host's `sys.executable` as the command and the script path as the only arg. A second pass starts NPX-based servers (currently a Playwright "browser" server) after a 3 s delay so the Python ones grab their slots first.

```python
# src/builtin_mcp.py
_BUILTIN_SERVERS = {
    "image_gen":  ("mcp_servers/image_gen_server.py",  "Built-in: Image Generation"),
    "memory":     ("mcp_servers/memory_server.py",     "Built-in: Memory"),
    "rag":        ("mcp_servers/rag_server.py",        "Built-in: RAG"),
    "email":      ("mcp_servers/email_server.py",      "Built-in: Email"),
}
```

`McpManager._connect_stdio` then does what any MCP client does: spin up `stdio_client`, open a `ClientSession`, `initialize()`, then `list_tools()` to discover what the server actually exposes. The discovered tools are cached in `self._tools[server_id]`, and an `AsyncExitStack` per server holds the stdio pipes and session for clean teardown. There is no special path for built-ins at this layer — they are just connections whose `server_id` happens to be `email`, `image_gen`, etc.

```text
+-----------------+       stdio        +----------------------------+
| McpManager      |  <-------------->  | python mcp_servers/email_  |
|  _sessions["email"]                  |   server.py                |
|  _tools["email"] = [list_emails,...] |   Server("email")          |
+-----------------+                    +----------------------------+
        ^                                          |
        | call_tool("mcp__email__list_emails",...)|
        |                                          v
        |                              imaplib -> data/app.db
        |                              SMTP    -> email_cache.db
```

The host marks these servers as built-ins purely by id membership in `is_builtin`, which is later used to enable auto-reconnect if the subprocess dies mid-call:

```python
# src/mcp_manager.py
def is_builtin(self, server_id: str) -> bool:
    return server_id.startswith("builtin_") or server_id in {
        "image_gen", "memory", "rag", "email",
    }
```

When `call_tool` raises against a built-in, `_reconnect_builtin` looks the server up in `_BUILTIN_SERVERS`, tears down the dead exit stack, re-spawns the subprocess, and retries the call once. Third-party servers get no such second chance.

Sources: [src/builtin_mcp.py:48-103](src/builtin_mcp.py), [src/mcp_manager.py:53-105](src/mcp_manager.py), [src/mcp_manager.py:196-295](src/mcp_manager.py), [src/mcp_manager.py:349-356](src/mcp_manager.py)

## The qualified-name trick — and a deliberate asymmetry

Every MCP tool is exposed to the model under a namespaced name `mcp__{server_id}__{tool_name}`. `McpManager.call_tool` parses that back into `server_id` and `tool_name` and dispatches to the right session. This is what lets the agent loop treat email tools, the browser tool, and a third-party Slack tool identically.

But `get_all_openai_schemas` and `get_tool_descriptions_for_prompt` deliberately *skip* the Python built-ins when generating OpenAI function-calling schemas:

```python
# src/mcp_manager.py
if self.is_builtin(server_id) and server_id != "builtin_browser":
    continue
```

The reason is that those four servers are also surfaced through Odysseus's own code-block tool format inside the agent prompt, so re-advertising them as JSON-schema function tools would double-list them. The NPX-based Playwright server — which is built-in in the sense of "ships with the product" but does not have hand-written agent-prompt entries — *does* go through function calling. So a builder reading the code finds two surfaces:

- For `email` / `image_gen` / `memory` / `rag`: the JSON-RPC plumbing is MCP, but the model sees a curated code-block tool description.
- For `builtin_browser` and every external server: the model sees a standard OpenAI function tool with the `mcp__...` qualified name.

That asymmetry is intentional and is the one thing easy to miss when porting another tool into this layout.

Sources: [src/mcp_manager.py:196-236](src/mcp_manager.py), [src/mcp_manager.py:297-330](src/mcp_manager.py), [src/mcp_manager.py:369-409](src/mcp_manager.py)

## What was *removed* tells you the design rule

A comment in `builtin_mcp.py` explains a recent refactor:

> `bash / python / filesystem / web_search` were folded into native in-process execution (`src/tool_execution.py:_direct_fallback`). Those trivial subprocess wrappers are gone.
>
> `image_gen / memory / rag / email` still run as stdio MCP servers — each carries hundreds of LOC of unique IMAP / HTTP / manager logic not worth duplicating into the native path right now.

The rule is roughly: if a tool is a thin shim over a stdlib call, keep it in-process; if it carries a body of unique logic that also benefits from process isolation (separate Python interpreter, can crash without taking the host down, can be hot-reconnected), keep it as an MCP server. Email is the clearest case — `email_server.py` is ~1,600 lines of IMAP folder resolution, multi-account routing, header decoding, MIME extraction, and SMTP send-with-Sent-folder-append.

Sources: [src/builtin_mcp.py:39-53](src/builtin_mcp.py), [mcp_servers/email_server.py:127-215](mcp_servers/email_server.py), [mcp_servers/email_server.py:771-833](mcp_servers/email_server.py)

## Subsystem-by-subsystem notes for builders

### Email server: multi-account by selector

`_load_config(account)` resolves an account selector by id, then exact-match against name/imap_user/from_address, then fuzzy match via `difflib.get_close_matches` at 0.72 cutoff. SMTP passwords are decrypted with `src.secret_storage.decrypt` before being handed to `smtplib` — falling back to raw ciphertext was a documented past bug. Port `993` is treated as implicit IMAP TLS; port `587` triggers `starttls()`; `465` uses `SMTP_SSL`. When no `account` is passed and 2+ accounts exist, `list_emails` *fans out across all accounts* and merges results sorted by `Date` header, prepending an `[EMAIL ACCOUNT CONTEXT: ...]` note so the model knows the result is merged.

Sources: [mcp_servers/email_server.py:88-215](mcp_servers/email_server.py), [mcp_servers/email_server.py:477-502](mcp_servers/email_server.py), [mcp_servers/email_server.py:1311-1384](mcp_servers/email_server.py)

### Image gen: provider-neutral by URL surgery

`image_gen_server.py` reads the *chat* model resolution from `src.ai_interaction._resolve_model`, then chops `/chat/completions` or `/v1/messages` off the URL and appends `/images/generations`. That keeps the server agnostic to which OpenAI-compatible backend a user configured. It enforces the per-model size whitelist (`gpt-image` accepts `1024x1024`, `1024x1536`, `1536x1024`, `auto`; `dall-e-3` accepts a different set), writes returned `b64_json` payloads under `data/generated_images/<uuid>.png`, and records a `GalleryImage` row so the host UI sees it. A 300 s read timeout is set explicitly because image models are slow.

Sources: [mcp_servers/image_gen_server.py:55-150](mcp_servers/image_gen_server.py)

### Memory: lazy init + dual-write to vector store

The memory server defers importing `MemoryManager` and `MemoryVectorStore` until the first `manage_memory` call (`_ensure_init`). When the vector store is healthy, every `add`/`edit`/`delete` is mirrored into it, but the mirror is best-effort — wrapped in `try/except` so a vector failure never blocks the JSON write. Search prefers `_memory_manager.get_relevant_memories(query, ..., threshold=0.05)` and falls back to substring filtering.

Sources: [mcp_servers/memory_server.py:27-44](mcp_servers/memory_server.py), [mcp_servers/memory_server.py:108-196](mcp_servers/memory_server.py)

### RAG: leans on host singletons

The RAG server is the thinnest of the four. It pulls `get_rag_manager()` and instantiates `PersonalDocsManager(PERSONAL_DIR, _rag_manager)` once, then delegates `list` / `add_directory` / `remove_directory` to those host objects. The only safeguards are `os.path.expanduser`, an `isdir` check before indexing, and a graceful "not available" path when either manager fails to load.

Sources: [mcp_servers/rag_server.py:25-44](mcp_servers/rag_server.py), [mcp_servers/rag_server.py:76-132](mcp_servers/rag_server.py)

## Lifecycle and admin surface

```mermaid
sequenceDiagram
    participant App as app startup
    participant Reg as register_builtin_servers
    participant Mgr as McpManager
    participant Proc as python mcp_servers/*.py
    participant Agent as agent loop
    App->>Reg: await register_builtin_servers(mcp_manager)
    Reg->>Mgr: connect_server(id, name, stdio, python, [script])
    Mgr->>Proc: spawn subprocess + open stdio pipes
    Proc-->>Mgr: initialize() ack
    Mgr->>Proc: list_tools()
    Proc-->>Mgr: [Tool(name=...), ...]
    Mgr-->>Reg: status=connected, tool_count=N
    Agent->>Mgr: call_tool("mcp__email__list_emails", args)
    Mgr->>Proc: session.call_tool("list_emails", args)
    Proc-->>Mgr: TextContent(...)
    Mgr-->>Agent: {stdout, exit_code}
    Note over Mgr,Proc: on exception, if is_builtin(id):<br/>_reconnect_builtin → respawn → retry once
```

Both built-in and user-added servers show up in `routes/mcp_routes.py`'s `GET /api/mcp/servers`, which calls `mcp_manager.get_server_status(srv.id)` and reports `tool_count`, `disabled_tool_count`, and connection status. Adding a stdio server through `POST /api/mcp/servers` is gated by `require_admin(request)` with a blunt comment in the code: registering a stdio server "is equivalent to executing arbitrary binaries on the host." Per-server tool blocklists live in `McpServer.disabled_tools` and are merged in via `_load_disabled_map()` before tools are advertised to the model.

A global kill switch — `ODYSSEUS_DISABLE_MCP=1` — short-circuits `register_builtin_servers` entirely.

Sources: [src/builtin_mcp.py:65-103](src/builtin_mcp.py), [src/mcp_manager.py:358-365](src/mcp_manager.py), [routes/mcp_routes.py:22-79](routes/mcp_routes.py), [routes/mcp_routes.py:81-103](routes/mcp_routes.py)

## What builders should take away

Three things worth borrowing:

1. **Same protocol for first- and third-party tools.** By writing local features as MCP servers, Odysseus avoids forking the agent's tool-calling code between "internal" and "external" paths. New built-ins drop into `_BUILTIN_SERVERS` and inherit reconnection, status, and admin UI for free.
2. **Process isolation as a reliability tool.** The auto-reconnect path for `is_builtin` servers means an IMAP socket dying or an image API hanging can be recovered without restarting the host. That is cheap to get when each server is its own Python subprocess, and hard to get when the same logic is inlined into the agent runtime.
3. **Keep the cheap stuff native.** Bash, Python eval, filesystem reads, and web search were *removed* from `mcp_servers/` and folded into in-process execution because the subprocess overhead bought nothing for a `subprocess.run` shim. The rule the codebase implicitly follows is: pay for MCP isolation only when the wrapped logic is heavy enough to be worth isolating.

The asymmetry around `get_all_openai_schemas` skipping Python built-ins is the most easily missed footgun — a contributor adding a fifth Python server should expect to either add it to the curated code-block tool list in the agent prompt or change `is_builtin` so the new server gets exposed through function calling.

Sources: [src/builtin_mcp.py:39-53](src/builtin_mcp.py), [src/mcp_manager.py:266-330](src/mcp_manager.py)