# OKF enrichment-agent CLI reference

> enrichment-agent enrich and visualize subcommands, BigQuery source flags, web crawl constraints, concept scoping, model selection, and environment variables.

- Repository: GoogleCloudPlatform/knowledge-catalog
- GitHub: https://github.com/GoogleCloudPlatform/knowledge-catalog
- Human docs: https://grok-wiki.com/public/docs/googlecloudplatform-knowledge-catalog-9cee6ee3cba5
- Complete Markdown: https://grok-wiki.com/public/docs/googlecloudplatform-knowledge-catalog-9cee6ee3cba5/llms-full.txt

## Source Files

- `okf/src/enrichment_agent/cli.py`
- `okf/pyproject.toml`
- `okf/src/enrichment_agent/sources/bigquery.py`
- `okf/src/enrichment_agent/web/fetcher.py`
- `okf/src/enrichment_agent/tools/web_tools.py`
- `okf/README.md`

---

---
title: OKF enrichment-agent CLI reference
description: enrichment-agent enrich and visualize subcommands, BigQuery source flags, web crawl constraints, concept scoping, model selection, and environment variables.
---

The `enrichment-agent` CLI produces [Open Knowledge Format](/open-knowledge-format) bundles from pluggable metadata sources. Today the only source is BigQuery (`--source bq`). The CLI exposes two subcommands: `enrich` runs a two-pass BQ-then-web enrichment pipeline into a bundle directory, and `visualize` renders a self-contained HTML graph viewer from an existing bundle.

Install the package from the `okf/` directory (see [Installation](/installation)), then invoke either entry point:

<CodeGroup>
```bash title="Console script"
enrichment-agent enrich --help
```

```bash title="Python module"
python -m enrichment_agent enrich --help
```
</CodeGroup>

## Enrichment pipeline

`enrich` runs in three phases:

1. **BigQuery pass** — For each concept the source advertises, an ADK agent reads metadata (and optional row samples) and writes one OKF markdown document per concept.
2. **Web pass** (optional) — When seed URLs are provided, a separate agent crawls documentation pages via the `fetch_url` tool and enriches existing concepts or mints `references/<slug>` docs.
3. **Index regeneration** — Auto-generated `index.md` files are written at each bundle directory level.

Use `--no-web` or omit all seed flags to run BQ-only. See [Enrichment workflows](/enrichment-workflows) and [Produce OKF bundles](/produce-okf-bundles) for end-to-end context.

## `enrich` subcommand

Enrich concepts from a source into an OKF bundle directory.

<RequestExample>
```bash
enrichment-agent enrich \
  --source bq \
  --dataset bigquery-public-data.ga4_obfuscated_sample_ecommerce \
  --web-seed-file samples/ga4_merch_store/seeds.txt \
  --out ./bundles/ga4
```
</RequestExample>

<ResponseExample>
```text
Enriched 12 concept(s) into ./bundles/ga4; web pass used 3 seed(s)
```
</ResponseExample>

On success the command prints a summary to stderr and exits `0`. With `-v` / `--verbose`, enrichment-agent loggers emit DEBUG-level tool-call detail while third-party loggers (`google`, `google_genai`, `google_adk`, `urllib3`, `httpx`) stay at WARNING.

### Source flags

<ParamField body="--source" type="string" required>
Source adapter to use. Currently only `bq` (BigQuery) is supported.
</ParamField>

<ParamField body="--dataset" type="string" required>
BigQuery dataset identifier in `project.dataset` form (for example `bigquery-public-data.ga4_obfuscated_sample_ecommerce`). Required when `--source bq`; the CLI exits if omitted.
</ParamField>

<ParamField body="--billing-project" type="string">
Google Cloud project ID billed for BigQuery API calls and query bytes. Defaults to the Application Default Credentials default project. Public datasets are readable, but the caller's project is billed for queries.
</ParamField>

<ParamField body="--out" type="path" required>
Bundle root directory. Created if it does not exist. Each concept becomes a `.md` file under this tree.
</ParamField>

#### BigQuery concepts

`BigQuerySource` discovers concepts automatically:

| Concept type | ID pattern | Notes |
|---|---|---|
| Dataset | `datasets/<dataset_id>` | One per `--dataset` |
| Table (singleton) | `tables/<table_id>` | Tables without a shard suffix |
| Table (sharded family) | `tables/<prefix>` | Tables matching `prefix` + 6–8 digit suffix collapse into one wildcard concept; enrichment reads the last shard as representative |

### Concept scoping

<ParamField body="--concept" type="string">
Enrich only the named concept ID. Repeatable. IDs are slash-separated path segments matching bundle layout (for example `tables/events_` for a sharded GA4 events family, or `datasets/ga4_obfuscated_sample_ecommerce`). Unknown IDs raise `ValueError` before enrichment starts.
</ParamField>

<RequestExample>
```bash
enrichment-agent enrich \
  --source bq \
  --dataset bigquery-public-data.ga4_obfuscated_sample_ecommerce \
  --concept tables/events_ \
  --no-web \
  --out ./bundles/ga4
```
</RequestExample>

### Model selection

<ParamField body="--model" type="string">
Gemini model ID passed to both the BQ and web ADK agents and to index regeneration. Default: `gemini-flash-latest`.
</ParamField>

Model credentials are not CLI flags. Configure one of the provider paths documented in [Installation](/installation):

| Provider | Environment variables |
|---|---|
| Google AI Studio | `GEMINI_API_KEY` |
| Vertex AI | `GOOGLE_GENAI_USE_VERTEXAI=true`, `GOOGLE_CLOUD_PROJECT`, `GOOGLE_CLOUD_LOCATION` |

BigQuery reads metadata via `gcloud auth application-default login` and the ADC default project unless `--billing-project` overrides it.

### Web crawl flags

The web pass is skipped when `--no-web` is set or when no seeds are collected. Seeds come from `--web-seed` (repeatable) and/or `--web-seed-file` (repeatable; one URL per line, `#` comments allowed). Duplicate URLs are deduplicated in order.

<ParamField body="--web-seed" type="string">
Individual seed URL. Repeatable.
</ParamField>

<ParamField body="--web-seed-file" type="path">
Path to a text file with one seed URL per line. Lines after `#` are comments. Repeatable; all files are merged.
</ParamField>

<ParamField body="--no-web" type="boolean">
Skip the web pass entirely. Seeds are ignored.
</ParamField>

<ParamField body="--web-max-pages" type="integer">
Hard cap on pages the web agent may fetch in one run. Default: `100`. Enforced inside the `fetch_url` tool; when the budget is spent, further fetches return `"max_pages reached"`.
</ParamField>

<ParamField body="--web-max-depth" type="integer">
Maximum hop distance from any seed URL. Seeds are depth `0`; their outbound links are depth `1`, and so on. Default: `2`. URLs beyond this depth are rejected.
</ParamField>

<ParamField body="--web-allowed-host" type="string">
Extra hostname the crawler may fetch beyond seed hostnames. Repeatable. By default, only hostnames extracted from seed URLs are allowed.
</ParamField>

<ParamField body="--web-allowed-path-prefix" type="string">
Only fetch URLs whose path starts with one of these prefixes (for example `/docs/`). Repeatable. Default: no path restriction.
</ParamField>

<ParamField body="--web-denied-path-substring" type="string">
Reject URLs whose path contains any of these substrings (for example `/login`, `/pricing`). Repeatable.
</ParamField>

#### Crawl enforcement

Constraints are enforced inside `fetch_url`, not by prompt alone. A rejected fetch returns an `error` field instead of page content; the agent should not retry the same URL.

| Constraint | Behavior |
|---|---|
| Scheme | Only `http` and `https` |
| Host | Must be in allowed-host set (seed hosts ∪ `--web-allowed-host`) |
| Path prefix | Must match at least one `--web-allowed-path-prefix` when any are set |
| Denied substring | Rejected if path contains any `--web-denied-path-substring` |
| Depth | Must be reachable from a seed via followed links and ≤ `--web-max-depth` |
| Reachability | URLs not returned as links from a fetched page are rejected ("not reachable from a seed") |
| Deduplication | Already-fetched URLs are rejected |
| Page budget | Rejected when `fetched_count >= --web-max-pages` |
| Content type | HTML only; non-HTML responses fail |
| Page size | Markdown body truncated at 40 KiB per page |
| Timeout | 10-second fetch timeout per page |

### Logging

<ParamField body="-v, --verbose" type="boolean">
Set `enrichment_agent` loggers to DEBUG for full tool-call arguments and responses.
</ParamField>

## `visualize` subcommand

Generate a self-contained HTML graph viewer from an OKF bundle. No backend or install is required on the viewing side. See [Visualize OKF bundles](/visualize-okf-bundles) for viewer features.

<RequestExample>
```bash
enrichment-agent visualize \
  --bundle ./bundles/crypto_bitcoin \
  --out /tmp/btc.html \
  --name "Bitcoin OKF"
```
</RequestExample>

<ResponseExample>
```text
Wrote 18 concept(s), 24 edge(s), 2847192 bytes → /tmp/btc.html
```
</ResponseExample>

<ParamField body="--bundle" type="path" required>
Path to the bundle root directory. Raises `FileNotFoundError` if the directory does not exist.
</ParamField>

<ParamField body="--out" type="path">
Output HTML path. Default: `<bundle>/viz.html`.
</ParamField>

<ParamField body="--name" type="string">
Display name shown in the viewer header. Default: bundle directory name.
</ParamField>

The generator walks all `*.md` files except `index.md`, parses OKF frontmatter, extracts cross-links, and embeds the graph as JSON in a single HTML file with bundled CSS and JavaScript.

## Quick recipes

<Steps>
<Step title="BQ-only enrichment">
Run metadata enrichment without web grounding:

```bash
enrichment-agent enrich \
  --source bq \
  --dataset <project>.<dataset> \
  --no-web \
  --out ./bundles/<name>
```
</Step>

<Step title="BQ plus web enrichment">
Provide documentation seeds for the web pass:

```bash
enrichment-agent enrich \
  --source bq \
  --dataset <project>.<dataset> \
  --web-seed-file <path/to/seeds.txt> \
  --web-max-pages 50 \
  --out ./bundles/<name>
```
</Step>

<Step title="Visualize the bundle">
After enrichment completes:

```bash
enrichment-agent visualize --bundle ./bundles/<name>
```

Open `<bundle>/viz.html` in a browser.
</Step>
</Steps>

Copy-paste dataset recipes with exact seed files and expected outputs are in [OKF bundle recipes](/okf-bundle-recipes).

## Environment variables

| Variable | Used by | Purpose |
|---|---|---|
| `GEMINI_API_KEY` | Gemini via ADK | API key for Google AI Studio |
| `GOOGLE_GENAI_USE_VERTEXAI` | Gemini via ADK | Set `true` to route through Vertex AI |
| `GOOGLE_CLOUD_PROJECT` | Vertex AI | GCP project for model calls |
| `GOOGLE_CLOUD_LOCATION` | Vertex AI | Region (for example `us-central1`) |
| ADC (via `gcloud auth application-default login`) | BigQuery client | Metadata reads and row sampling; billing project from ADC unless `--billing-project` is set |

No enrichment-agent-specific environment variables exist beyond the Gemini/Vertex configuration consumed by `google-adk` and `google-genai`.

## Exit codes and errors

| Condition | Result |
|---|---|
| Missing `--dataset` for `--source bq` | Exit `1` with message `--dataset is required for --source bq` |
| Unknown `--source` value | Exit `1` with `Unknown source: …` |
| Invalid `--concept` ID (unknown to source) | `ValueError: Unknown concept(s): …` |
| Invalid dataset format (not `project.dataset`) | `ValueError` from `BigQuerySource` |
| Missing bundle directory for `visualize` | `FileNotFoundError` |

## Distinction from catalog enrichment agent

This CLI (`enrichment-agent` in the `okf/` package) produces standalone OKF bundle directories from BigQuery. The separate catalog enrichment agent (`agent_runner.py` in `agents/enrichment/`) runs table, doc, and context_overlay modes against Knowledge Catalog workspaces. Flag reference for that agent is on [Enrichment agent flags](/enrichment-agent-flags).

## Related pages

<Card href="/installation" title="Installation" icon="download">
Prerequisites, package install, and credential setup for BigQuery and Gemini.
</Card>

<Card href="/produce-okf-bundles" title="Produce OKF bundles" icon="package">
End-to-end workflow for BQ-then-web enrichment into a versionable bundle.
</Card>

<Card href="/visualize-okf-bundles" title="Visualize OKF bundles" icon="network">
Graph viewer features: force-directed layout, detail panels, backlinks, and search.
</Card>

<Card href="/okf-bundle-recipes" title="OKF bundle recipes" icon="book-open">
GA4, Stack Overflow, and Bitcoin sample commands with seed files.
</Card>

<Card href="/troubleshooting" title="Troubleshooting" icon="life-buoy">
Auth, billing, web crawl cap, and model credential failures.
</Card>