# Tool Plugin Model: Discovery, Secrets, & the centaur_sdk
> How tools are discovered (Python files in tools/ or overlays), loaded by tool_manager.py, and exposed to sandboxes. The ToolContext / secret() resolution chain (tool context → pluggable backend → default). The SecretMode enum (replace vs inject). How tool authors import centaur_sdk and never see raw credentials.
- Repository: paradigmxyz/centaur
- GitHub: https://github.com/paradigmxyz/centaur
- Human wiki: https://grok-wiki.com/public/wiki/paradigmxyz-centaur-57fc6b2755e2
- Complete Markdown: https://grok-wiki.com/public/wiki/paradigmxyz-centaur-57fc6b2755e2/llms-full.txt
## Source Files
- `services/api/api/tool_manager.py`
- `centaur_sdk/tool_sdk.py`
- `centaur_sdk/backends/base.py`
- `centaur_sdk/backends/registry.py`
- `centaur_sdk/tests/test_tool_sdk.py`
---
Relevant source files
The following files were used as context for generating this wiki page:
- [services/api/api/tool_manager.py](services/api/api/tool_manager.py)
- [centaur_sdk/tool_sdk.py](centaur_sdk/tool_sdk.py)
- [centaur_sdk/__init__.py](centaur_sdk/__init__.py)
- [centaur_sdk/backends/base.py](centaur_sdk/backends/base.py)
- [centaur_sdk/backends/registry.py](centaur_sdk/backends/registry.py)
- [centaur_sdk/backends/stub.py](centaur_sdk/backends/stub.py)
- [centaur_sdk/backends/env.py](centaur_sdk/backends/env.py)
- [centaur_sdk/tests/test_tool_sdk.py](centaur_sdk/tests/test_tool_sdk.py)
- [tools/research/harmonic/client.py](tools/research/harmonic/client.py)
- [tools/research/harmonic/pyproject.toml](tools/research/harmonic/pyproject.toml)
# Tool Plugin Model: Discovery, Secrets, & the centaur_sdk
Centaur's tool plugin model lets external contributors add new capabilities by dropping a Python package into a `tools/` directory (or a configured overlay). The API service discovers, loads, and registers these tools at startup — and hot-reloads them on demand — without any manual wiring. The other half of the model is credential isolation: tool code never sees raw API keys. Instead it calls `secret()` from `centaur_sdk`, receives a placeholder token, and iron-proxy swaps that token for the real credential on the outbound wire. This page covers how those two halves fit together.
---
## Tool Discovery
### Directory Layout
`ToolManager` is initialized with one or more `tools_dirs`. Each base directory is scanned for Python packages at load time. The scanner supports one level of **category subdirectories**: if a child folder has no `pyproject.toml` it is treated as a category folder and its children are each treated as a tool candidate (e.g. `tools/research/harmonic/`).
```
tools/
slack/ ← top-level tool (has pyproject.toml)
research/ ← category folder (no pyproject.toml)
harmonic/ ← tool inside category
crunchbase/
```
Hidden and underscore-prefixed directories (`.foo`, `_bar`) are skipped. When the same tool name appears in two different `tools_dirs`, the later one silently shadows the earlier one — this is the intended mechanism for private overlays.
Sources: [services/api/api/tool_manager.py:1376-1485]()
### pyproject.toml Manifest
Every tool package declares itself via a standard `pyproject.toml`. The `[project]` table supplies `name` (used as the tool's identifier) and `description`. The `[tool.centaur]` table carries runtime metadata:
| Key | Meaning |
|-----|---------|
| `module` | Python file to import (default: `client.py`) |
| `secrets` | Required credentials (tool unavailable if missing) |
| `optional_secrets` | Credentials used when present but not gating availability |
| `hosts` | Tool-level fallback host scope for secret entries |
| `timeout_s` / `timeout_env` | Per-tool call timeout override |
| `type = "persona"` | Marks the package as a persona, not a callable tool |
Example from `tools/research/harmonic/pyproject.toml`:
```toml
[tool.centaur]
module = "client.py"
secrets = [{type = "http", name = "HARMONIC_API_KEY", match_headers = ["apikey"], hosts = ["api.harmonic.ai"]}]
```
Sources: [tools/research/harmonic/pyproject.toml:16-18]()
### Module Loading
`ToolManager._load_tool()` registers the tool directory as a synthetic Python package under the `shared.tools_runtime.` namespace, then imports the configured `module` file using `importlib.util`. This allows relative imports within a tool package to work correctly.
The key sequence during loading:
1. A bare `ToolContext(name=name, secrets={})` is created and pushed onto a `ContextVar` via `set_tool_context(ctx)`.
2. The module is executed via `spec.loader.exec_module(module)` while that context is live.
3. `_collect_methods(module)` calls the module's `_client()` factory once, then enumerates every public, non-lifecycle, non-property method on the returned instance.
4. `reset_tool_context(token)` pops the context whether or not loading succeeded.
The `_client()` pattern is the required convention: each tool module must expose this callable factory. Methods named `close`, `connect`, `disconnect`, or `shutdown` are excluded from the exposed method list (`_LIFECYCLE_METHODS`).
Sources: [services/api/api/tool_manager.py:1663-1763]()
---
## ToolContext and the secret() Resolution Chain
### ToolContext Dataclass
`ToolContext` is a lightweight dataclass defined in `centaur_sdk/tool_sdk.py`:
```python
@dataclass
class ToolContext:
name: str
secrets: dict[str, str] = field(default_factory=dict)
thread_key: str | None = None
container_id: str | None = None
```
It is stored in a `ContextVar[ToolContext]` named `_tool_ctx`, which makes it coroutine- and thread-safe: each invocation can carry its own context without interference.
Sources: [centaur_sdk/tool_sdk.py:19-27]()
### secret() Resolution Order
When tool code calls `secret("SOME_KEY")`, the function walks three sources in strict priority order:
```
1. ToolContext.secrets dict (set by ToolManager at call time)
↓ miss
2. Pluggable backend (SecretBackend from registry)
↓ miss
3. Caller-supplied default (or KeyError with tool name included)
```
```python
def secret(key: str, default: str | None = None) -> str:
try:
ctx = _tool_ctx.get()
val = ctx.secrets.get(key)
if val is not None:
return val
except LookupError:
pass
from centaur_sdk.backends.registry import get_backend
val = get_backend().get_sync(key)
if val is not None:
return val
if default is not None:
return default
raise KeyError(f"Missing secret '{key}'{ctx_name}")
```
Sources: [centaur_sdk/tool_sdk.py:47-76]()
The test suite verifies each tier explicitly: tool context wins over backend, backend wins over default, and all-miss raises a `KeyError` that includes the tool name for easy diagnosis.
Sources: [centaur_sdk/tests/test_tool_sdk.py:27-64]()
### How ToolManager Populates ToolContext at Call Time
When a tool method is invoked, `call_tool()` (or `call_tool_raw()`) resolves placeholder values for all `replace`-mode `HttpSecret` entries declared by the tool, then constructs a fresh `ToolContext` that includes those placeholders plus any sandbox claims:
```python
resolved = await _resolve_secrets(all_secrets)
ctx = ToolContext(
name=lt.name,
secrets={**lt.ctx.secrets, **resolved},
thread_key=sandbox_claims.get("thread_key") if sandbox_claims else None,
container_id=sandbox_claims.get("container_id") if sandbox_claims else None,
)
token = set_tool_context(ctx)
```
`_resolve_secrets` only returns values for replace-mode `HttpSecret` entries (the placeholder token). Inject-mode, `GcpAuthSecret`, `OAuthTokenSecret`, and `PgDsnSecret` are invisible to tool code — those are handled entirely by iron-proxy.
Sources: [services/api/api/tool_manager.py:838-850](), [services/api/api/tool_manager.py:1885-1915]()
---
## The SecretMode Enum: replace vs inject
`SecretMode` is a two-value enum that controls how iron-proxy applies an HTTP credential:
| Mode | What the tool sees | What iron-proxy does |
|------|--------------------|----------------------|
| `REPLACE` | The placeholder token (e.g. `"HARMONIC_API_KEY"`) written into a header/path/query | Scans the outbound request for the token and swaps it for the real value from `secret_ref` |
| `INJECT` | Nothing — the credential is never in the tool's scope | Adds the resolved credential directly to the request (header or query param) before it leaves the network |
```python
class SecretMode(str, Enum):
REPLACE = "replace"
INJECT = "inject"
```
Sources: [services/api/api/tool_manager.py:65-73]()
The `HttpSecret` dataclass enforces that each mode declares the right fields: replace-mode requires at least one of `match_headers`, `match_path`, or `match_query`; inject-mode requires exactly one of `inject_header` or `inject_query_param`, and `inject_formatter` (a Go template like `Bearer {{ .Value }}`) is only valid alongside `inject_header`. Cross-mode fields are rejected at parse time.
Sources: [services/api/api/tool_manager.py:560-622]()
---
## Secret Types Beyond HTTP
`SecretDef` is a union of five types, each handled completely by iron-proxy with no raw credential reaching the sandbox:
| Type | `SecretDef` class | How it works |
|------|--------------------|--------------|
| HTTP header/query/path | `HttpSecret` | replace or inject mode as above |
| GCP service-account | `GcpAuthSecret` | iron-proxy mints OAuth2 tokens from a keyfile |
| OAuth2 token exchange | `OAuthTokenSecret` | iron-proxy runs the grant flow (refresh_token, client_credentials, password, or jwt_bearer) and injects Bearer tokens |
| Postgres DSN | `PgDsnSecret` | iron-proxy exposes a local TCP listener; sandbox connects to that, iron-proxy forwards to the upstream DSN |
| Per-request HMAC | `HmacSignSecret` | iron-proxy computes an HMAC signature and adds the resulting headers |
`_parse_secret()` in `tool_manager.py` maps each `type` key in `pyproject.toml` to the appropriate dataclass, validating required and optional fields with descriptive `ValueError` messages.
Sources: [services/api/api/tool_manager.py:647-820]()
---
## The Pluggable Backend System
### SecretBackend ABC
`centaur_sdk.backends.base.SecretBackend` is the abstract interface all backends implement:
```python
class SecretBackend(ABC):
@abstractmethod
async def get(self, key: str) -> str | None: ...
@abstractmethod
async def list_keys(self) -> list[str]: ...
def get_sync(self, key: str) -> str | None:
# runs in a background thread when called from an event loop
```
`get_sync` bridges sync tool code and async backends without blocking the event loop: it uses `asyncio.run()` outside a loop and `ThreadPoolExecutor` inside one.
Sources: [centaur_sdk/backends/base.py:9-34]()
### Built-in Backends
| Class | Module | Behavior |
|-------|--------|----------|
| `StubBackend` | `centaur_sdk.backends.stub` | Returns the key name as the value. Server-mode default. Falls back to env var if key is present (needed for Postgres DSNs that cannot go through firewall injection). |
| `EnvBackend` | `centaur_sdk.backends.env` | Returns `os.environ.get(key)`. CLI mode only — **banned in server mode**. |
Sources: [centaur_sdk/backends/stub.py:17-33](), [centaur_sdk/backends/env.py:10-17]()
### Registry
`centaur_sdk.backends.registry` holds a module-level singleton `_backend`. On first use, `get_backend()` calls `auto_configure()`, which installs `StubBackend`. This is the invariant that prevents real credentials from ever being resolvable inside the API process:
> **Do not use `EnvBackend` here.** Real secrets must never be resolvable inside the API process. See README.md § Security Architecture, invariant S1.
Sources: [centaur_sdk/backends/registry.py:24-46]()
CLI tools explicitly call `registry.configure(EnvBackend())` before using `secret()` so local development works without a running iron-proxy.
---
## Infra-Level Secrets
`ToolManager` maintains a hardcoded `_INFRA_SECRETS` class variable for first-party AI provider keys (Anthropic, OpenAI, xAI, Gemini, AMP, GitHub, Slack). These are always included in the secrets map handed to iron-proxy via `collect_secrets()`, ahead of any tool-declared secrets.
```python
_INFRA_SECRETS: ClassVar[list[HttpSecret]] = [
HttpSecret(
name="ANTHROPIC_API_KEY",
secret_ref="ANTHROPIC_API_KEY",
hosts=("api.anthropic.com",),
match_headers=("X-Api-Key",),
),
...
]
```
Sources: [services/api/api/tool_manager.py:1594-1648]()
---
## End-to-End Flow
```
┌─────────────────────────────────────────────────────────────┐
│ tools/research/harmonic/ │
│ pyproject.toml → [tool.centaur] secrets = [...] │
│ client.py → from centaur_sdk import secret │
│ def _client() -> HarmonicClient: ... │
└────────────────────┬────────────────────────────────────────┘
│ ToolManager.discover()
▼
┌─────────────────────────────────────────────────────────────┐
│ ToolManager._load_tool() │
│ 1. parse secrets from pyproject.toml │
│ 2. set_tool_context(ToolContext(name, secrets={})) │
│ 3. importlib exec_module(client.py) │
│ 4. _collect_methods(_client()) → LoadedTool │
│ 5. reset_tool_context(token) │
└────────────────────┬────────────────────────────────────────┘
│ HTTP POST /tools/harmonic/search
▼
┌─────────────────────────────────────────────────────────────┐
│ ToolManager.call_tool() │
│ 1. _resolve_secrets() → {"HARMONIC_API_KEY": "HARMONIC_API_KEY"} │
│ 2. set_tool_context(ToolContext(secrets=resolved, ...)) │
│ 3. method.fn(**args) under asyncio.wait_for(timeout) │
│ 4. reset_tool_context(token) │
└────────────────────┬────────────────────────────────────────┘
│ outbound HTTPS to api.harmonic.ai
▼
┌─────────────────────────────────────────────────────────────┐
│ iron-proxy (firewall) │
│ sees "apikey: HARMONIC_API_KEY" in request header │
│ resolves HARMONIC_API_KEY from env/1Password │
│ replaces header with real credential before forwarding │
└─────────────────────────────────────────────────────────────┘
```
The invariant is: tool code receives a string (`"HARMONIC_API_KEY"`) that is useless outside the iron-proxy perimeter. The real key is never present in the Python process.
---
## What Tool Authors Import
The `centaur_sdk` package surface is intentionally minimal. Everything a tool author needs is exported from `centaur_sdk/__init__.py`:
| Symbol | Purpose |
|--------|---------|
| `secret(key, default?)` | Resolve a credential via the three-tier chain |
| `current_thread_key()` | Get the active sandbox thread key |
| `save_attachment(...)` | Persist binary output scoped to the current thread |
| `save_attachment_from_path(...)` | Persist a local file as a thread attachment |
| `ToolContext` | Dataclass; authors read it via `get_tool_context()` but rarely construct it |
| `Table`, `render_text_table` | CLI formatting helpers |
Sources: [centaur_sdk/__init__.py:12-34]()
Tool code never imports `ToolManager`, never sets `ToolContext`, and never touches the backend registry. The call to `set_tool_context` / `reset_tool_context` is the exclusive responsibility of `ToolManager`.
---
## Failure Modes and Invariants
| Scenario | Behavior |
|----------|----------|
| Tool's `module` file missing | Logged as `tool_module_missing`, tool skipped; `ToolManager.load_failures` records it |
| `secret()` called before any context set | Falls through to backend (StubBackend in server mode), which returns the key name as the value — a safe stub, not a crash |
| Required secret unresolvable | `GET /tools` omits the tool from the listing; `GET /tools/` returns HTTP 404 |
| Catch-all or IP host patterns in secrets | `tool_invalid_host` warning logged; tool still loads but the misconfigured secret is flagged |
| Tool shadows another in a later overlay dir | `tool_shadowed` logged; later entry wins silently |
| Tool method times out | `asyncio.TimeoutError` caught; returns `{"error": "Tool call timed out after Xs"}` |
| `EnvBackend` installed in server mode | Ruff per-file-ignore rule in pyproject.toml prevents the import; policy enforced at lint time, not runtime |
The overall design ensures that a misbehaving tool — including one that leaks its own `ToolContext` — can only ever leak a placeholder string that iron-proxy would reject from an unscoped origin.
Sources: [centaur_sdk/backends/registry.py:24-37](), [services/api/api/tool_manager.py:1540-1589]()