# Search spans
> Semantic search over embedded main-conversation LLM spans via GET /v1/search/spans, tapesctl search, and the MCP search tool.
- Repository: papercomputeco/tapes
- GitHub: https://github.com/papercomputeco/tapes
- Human docs: https://grok-wiki.com/public/docs/papercomputeco-tapes-020596f21750
- Complete Markdown: https://grok-wiki.com/public/docs/papercomputeco-tapes-020596f21750/llms-full.txt
## Source Files
- `api/search_spans_handler.go`
- `pkg/spanembed/spanembed.go`
- `pkg/embedworker/worker.go`
- `api/mcp/search.go`
- `docs/search.md`
- `pkg/embeddings/embedder.go`
---
---
title: "Search spans"
description: "Semantic search over embedded main-conversation LLM spans via GET /v1/search/spans, tapesctl search, and the MCP search tool."
---
`GET /v1/search/spans` on the read API (`:8081`) embeds the query text and runs pgvector similarity over the `span_embeddings` projection. Hits are individual **main-conversation LLM spans** (`kind=llm`, `call_kind=main`), each carrying session, trace (turn), span id, score, turn prompt, snippet, model, and start time. There is no `/v1/search` route, and search does not return sessions, tool/event spans, shadow LLM calls, or in-memory merkle content.
`tapesctl search` and the MCP tool `search` call the same span searcher. `tapesctl` is the client binary (separate repository); this server owns the embed worker and the HTTP/MCP surfaces.
Ingest never embeds. Capture writes `raw_turns` on `:8082`; the deriver projects spans; the embed worker writes vectors. A derived span is invisible to search until an embed pass has stored it.
## What a hit is
Each result is one span, not a session. Clients jump to the matched turn with `session_id` + `trace_id` + `span_id`.
| Field | Meaning |
| --- | --- |
| `trace_id` | Derived turn the span belongs to |
| `span_id` | Matched span |
| `session_id` | Session when attribution exists; omitted when the span derived without one |
| `score` | Similarity: `1 -` pgvector cosine distance (`<=>`) of the best-matching chunk |
| `user_prompt` | Prompt of that turn (HTTP always includes the key, including `""`) |
| `snippet` | Preview of the span's embedded delta text, truncated at 280 runes plus `…` |
| `model` | Model that produced the span (not the embedding model) |
| `started_at` | Span start time |
Search is scoped to this deployment's single tenant (`00000000-0000-0000-0000-000000000000`). A caller-supplied `X-Tapes-Org-Id` header is ignored. Header-less MCP search uses the same nil-org bucket.
## What gets embedded
Only spans that pass all of these enter `span_embeddings`:
- `kind = llm` and `call_kind = main` (permission checks, title generation, and other shadow calls are excluded)
- Rendered delta-only text is non-empty: text blocks from stored `input` plus `output`
- Tool payloads, thinking, images, and harness tags (``, hook context, command framing) are stripped before hashing and embedding
Embeddings are keyed by `(org_id, trace_id, span_id)` and a SHA-256 of the rendered text. Unchanged content under the same embedding model is skipped. Switching `embedding.model` re-embeds. A re-derive that prunes or reclassifies a span orphans its rows; the next pass deletes them.
Oversized text is split into chunk rows (`chunk_idx` 0..N-1). Search over-fetches `top_k * 4` nearest chunks, then collapses to one hit per span using the best chunk. Rendered text above ~1 MiB (`DefaultMaxTextBytes`) is recorded as a deterministic `too_large` failure and not retried until content or model changes. Transient provider errors stay un-embedded and retry on the next pass.
`embedding.model` and `embedding.dimensions` must match the pgvector column. `EnsureSchema` fail-fasts if an existing `span_embeddings` table was created with a different size; pgvector cannot resize the column in place.
```mermaid
flowchart LR
subgraph writePath [Write path]
Ingest["ingest :8082"]
Raw["raw_turns"]
Derive["derive-worker"]
Spans["spans llm + main"]
EmbedW["embed-worker"]
Vec["span_embeddings"]
Ingest --> Raw --> Derive --> Spans --> EmbedW --> Vec
end
subgraph readPath [Read path :8081]
Q["Embed query"]
API["GET /v1/search/spans"]
API --> Q --> Vec
end
Provider["embedding.provider"]
EmbedW --> Provider
Q --> Provider
```
## Prerequisites
`tapes local up` provisions PostgreSQL with pgvector and the default Ollama model `embeddinggemma` (768 dimensions). Defaults:
| Key | Default |
| --- | --- |
| `embedding.provider` | `ollama` |
| `embedding.target` | `http://localhost:11434` |
| `embedding.model` | `embeddinggemma` |
| `embedding.dimensions` | `768` |
Switch provider with `tapes config set embedding.provider openai` and store a key via `tapes auth openai` or `OPENAI_API_KEY`. See [Configure embeddings](/configure-embeddings).
`tapes serve` runs proxy, read API, ingest, derive worker, and — by default — an in-process embed loop (10s interval, one pass at startup). Disable embedding with `--embed-spans=false`.
```bash
tapesctl config set tapes-url http://localhost:8081
```
Search talks to `:8081`, not ingest `:8082`.
Seed or capture, then confirm sessions exist (`tapesctl sessions list`) and that traces/spans have been derived.
## Search with tapesctl
```bash title="Human-readable hits"
tapesctl search "how was authentication fixed?"
tapesctl search "logging configuration" --top 10
```
```bash title="Session ids only"
tapesctl search "Charm CLI patterns" --quiet --top 3
```
`--top` maps to `top_k` (default 5). `--quiet` is a pipe format, not a log level: one unique session id per line, in score order. Empty results are not an error: non-quiet prints `No results found.` and exits 0; quiet prints nothing and exits 0.
Quiet output composes with skill generation:
```bash
tapesctl skill generate $(tapesctl search "Charm CLI" --quiet --top 1) \
--name charm-patterns
```
## HTTP: GET /v1/search/spans
:::endpoint GET /v1/search/spans Embed the query and return the nearest main LLM spans
OpenAPI operation `searchSpans`, tag `search`. Compiled into `GET /openapi` on `:8081`.
Search text. Empty or missing returns 400: `query parameter is required`.
Maximum hits. Default `5`. Must be a positive integer or the handler returns 400: `top_k must be a positive integer`.
Echo of the request query.
Hits in distance order. Each object uses the fields in [What a hit is](#what-a-hit-is).
`len(results)`.
:::
```bash
curl --get http://localhost:8081/v1/search/spans \
--data-urlencode 'query=how was authentication fixed?' \
--data-urlencode 'top_k=5'
```
```json
{
"query": "how was authentication fixed?",
"count": 1,
"results": [
{
"trace_id": "trc_req1",
"span_id": "llm_req1",
"session_id": "5b6f0f8e-2c3a-4ec0-9b6e-000000000001",
"score": 0.91,
"user_prompt": "fix the retry backoff",
"snippet": "set max-poll-backoff to 30s",
"model": "claude-sonnet-4-5",
"started_at": "2026-06-01T12:00:00Z"
}
]
}
```
Errors use `{"error":"..."}`.
| Status | When | Body |
| --- | --- | --- |
| 400 | Missing `query`, or `top_k` not a positive integer | `query parameter is required` or `top_k must be a positive integer` |
| 500 | Query embed failed, or the store query failed | `failed to embed query: …` or the store error |
| 503 | Embedder or span store not wired on this process | `span search is not configured: embedder and span embedding store are required` |
| 503 | `span_embeddings` table does not exist yet | `span embeddings not initialized: run the embed pass (tapes serve embed-worker or tapes dev embed-spans)` |
A standalone `tapes serve api` constructs the store without creating the table. The process boots; search stays 503 until a writer (`tapes serve`, `tapes serve embed-worker`, or `tapes dev embed-spans`) has run `EnsureSchema`.
## MCP tool: search
Streamable HTTP MCP is at `http://localhost:8081/v1/mcp`. While search is still a core tool (slated for cassette extraction), it is registered only when both `Embedder` and `SpanSearcher` are configured. Cassette tools remain available if search is omitted.
| Field | Value |
| --- | --- |
| Name | `search` |
| Required | `query` string |
| Optional | `top_k` integer, default `5` |
The tool embeds the query and calls the same `SpanSearcher` as HTTP. Structured output matches `SpanSearchOutput` and is also returned as a JSON text block. Header-less calls use the nil-org tenant. Embed or store failures are MCP tool errors (`IsError`), including `ErrNotInitialized` when the table is missing.
```json
{
"query": "how was logging configured?",
"top_k": 3
}
```
## Keep the projection current
| Mode | Command | Behavior |
| --- | --- | --- |
| All-in-one | `tapes serve` | In-process embed loop, 10s interval, pass at startup (unless `--embed-spans=false`) |
| Split writer | `tapes serve embed-worker` | Own process; default interval `1m`; pass at startup; never blocks derive |
| One-shot backfill | `tapes dev embed-spans` | Walks every eligible span once |
Split deployment:
```bash
tapes serve derive-worker --postgres "$TAPES_STORAGE_POSTGRES_DSN"
tapes serve embed-worker --postgres "$TAPES_STORAGE_POSTGRES_DSN"
tapes serve api --postgres "$TAPES_STORAGE_POSTGRES_DSN"
```
Standalone embed-worker knobs include `--interval`, `--batch-size` (default page 100), `--max-text-bytes`, `--org`, and embedding provider/model/dimensions. Infrastructure failures (database or embedder unreachable) do not crash the worker: they log one line and retry with jittered exponential backoff (cap 5 minutes). Per-span failures increment the pass report (`failed`, `poisoned`, `empty`, `chunked`) and leave the span for a later run.
On `tapes serve`, embed setup failure is a warning that disables search; the rest of the stack still starts.
## Troubleshooting
1. Confirm the read API: `tapes status` and `curl -sS http://localhost:8081/ping`.
2. Confirm derived data: `tapesctl sessions list`, then traces for a session. Search cannot invent hits from `raw_turns` alone.
3. Confirm the embedder: for Ollama, `curl http://localhost:11434/api/tags` and that `embedding.model` is present.
4. Confirm `embedding.model` / `embedding.dimensions` match the table (default `embeddinggemma` @ `768`).
5. Read the 503 body: missing embedder/store vs uninitialized table are different strings.
6. In a split deploy, confirm `tapes serve embed-worker` is running. `--embed-spans=false` on `tapes serve` leaves the table uncreated until a writer runs.
7. Empty `results` with 200 means no embedded span was similar enough — not a transport error.
## Next
Switch `embedding.provider`, set model and dimensions, store keys, run or disable the embed worker.
Streamable HTTP at `/v1/mcp`, cassette tools, and the legacy `search` tool.
Compiled `GET /openapi` on `:8081`, including `searchSpans`.
Run derive-worker and embed-worker as separate processes.
Pipe quiet search session ids into skill generation.
How `call_kind=main` and deterministic ids relate to a search hit.