# OKF bundle recipes

> Copy-paste enrichment recipes for GA4 merchandise store, Stack Overflow, and Bitcoin public datasets with seed files, exact commands, and expected bundle outputs.

- Repository: GoogleCloudPlatform/knowledge-catalog
- GitHub: https://github.com/GoogleCloudPlatform/knowledge-catalog
- Human docs: https://grok-wiki.com/public/docs/googlecloudplatform-knowledge-catalog-9cee6ee3cba5
- Complete Markdown: https://grok-wiki.com/public/docs/googlecloudplatform-knowledge-catalog-9cee6ee3cba5/llms-full.txt

## Source Files

- `okf/samples/ga4_merch_store/README.md`
- `okf/samples/stackoverflow/README.md`
- `okf/samples/crypto_bitcoin/README.md`
- `okf/samples/ga4_merch_store/seeds.txt`
- `okf/samples/stackoverflow/seeds.txt`
- `okf/bundles/stackoverflow/datasets/stackoverflow.md`
- `okf/bundles/crypto_bitcoin/index.md`

---

---
title: "OKF bundle recipes"
description: "Copy-paste enrichment recipes for GA4 merchandise store, Stack Overflow, and Bitcoin public datasets with seed files, exact commands, and expected bundle outputs."
---

The `okf/samples/` directory pairs three copy-paste enrichment recipes with checked-in reference bundles under `okf/bundles/`. Each recipe runs `enrichment-agent enrich` against a BigQuery public dataset, optionally seeds a two-pass web crawl from a `seeds.txt` file, and writes an OKF v0.1 bundle of markdown concept documents with YAML frontmatter and auto-generated `index.md` navigation files.

<Info>
All commands below assume your working directory is `okf/` in the Knowledge Catalog repository. Install the agent with `python3.13 -m venv .venv` and `.venv/bin/pip install --index-url https://pypi.org/simple/ -e .[dev]`.
</Info>

## Recipe comparison

| Recipe | BigQuery dataset | Seed file | Output directory (recipe) | Checked-in reference bundle | Enrichment pattern |
|--------|------------------|-----------|---------------------------|----------------------------|--------------------|
| GA4 Google Merchandise Store | `bigquery-public-data.ga4_obfuscated_sample_ecommerce` | `samples/ga4_merch_store/seeds.txt` | `./bundles/ga4_merch_store` | `bundles/ga4/` | Single sharded `events_` table plus GA4 export reference docs |
| Stack Overflow | `bigquery-public-data.stackoverflow` | `samples/stackoverflow/seeds.txt` | `./bundles/stackoverflow` | `bundles/stackoverflow/` | Many entity tables; one schema page enriches multiple concepts |
| Bitcoin (crypto) | `bigquery-public-data.crypto_bitcoin` | `samples/crypto_bitcoin/seeds.txt` | `./bundles/crypto_bitcoin` | `bundles/crypto_bitcoin/` | Four tightly related fact tables with cross-table foreign-key prose |

## Shared prerequisites

<Steps>
<Step title="Install the enrichment agent">

From `okf/`:

```bash
python3.13 -m venv .venv
.venv/bin/pip install --index-url https://pypi.org/simple/ -e .[dev]
```

</Step>

<Step title="Configure BigQuery access">

Public datasets are readable, but your project is billed for query bytes:

```bash
gcloud auth application-default login
gcloud config set project <your-billing-project>
```

</Step>

<Step title="Configure model credentials">

Use either AI Studio or Vertex AI:

<Tabs>
<Tab title="AI Studio">

```bash
export GEMINI_API_KEY=<your-key>
```

</Tab>
<Tab title="Vertex AI">

```bash
export GOOGLE_GENAI_USE_VERTEXAI=true
export GOOGLE_CLOUD_PROJECT=<your-project-id>
export GOOGLE_CLOUD_LOCATION=<region>
```

</Tab>
</Tabs>

</Step>
</Steps>

## GA4 Google Merchandise Store

The GA4 recipe targets the public GA4 e-commerce export from the Google Merchandise Store. The BQ pass produces dataset and table concepts; the web pass seeds canonical GA4 BigQuery Export documentation and may mint reference docs for metrics and joins.

### Seed file

`okf/samples/ga4_merch_store/seeds.txt` lists GA4 BigQuery Export URLs. Lines starting with `#` are comments; blank lines are ignored. The web agent crawls outward from each seed URL, following links it judges relevant, restricted to seed hostnames by default.

```text title="samples/ga4_merch_store/seeds.txt"
# GA4 BigQuery Export — top-level overview and index
https://support.google.com/analytics/answer/7029846

# GA4 BigQuery Export — schema reference (events, items, params)
https://support.google.com/analytics/answer/7029846?hl=en
```

### Enrich command

<RequestExample>

```bash title="GA4 enrichment"
.venv/bin/python -m enrichment_agent enrich \
    --source bq \
    --dataset bigquery-public-data.ga4_obfuscated_sample_ecommerce \
    --web-seed-file samples/ga4_merch_store/seeds.txt \
    --out ./bundles/ga4_merch_store
```

</RequestExample>

### Iteration flags

<ParamField body="--concept" type="string">
Scope enrichment to one concept id. For GA4, use `tables/events_` to iterate on the sharded events table only. Repeatable.
</ParamField>

<ParamField body="--no-web" type="boolean">
Skip the web pass and emit BQ-only concepts.
</ParamField>

<ParamField body="--web-max-pages" type="integer" default="100">
Hard cap on pages the web agent may fetch in one run.
</ParamField>

Smoke-run example:

```bash title="Single-concept BQ-only"
.venv/bin/python -m enrichment_agent enrich \
    --source bq \
    --dataset bigquery-public-data.ga4_obfuscated_sample_ecommerce \
    --concept tables/events_ \
    --no-web \
    --out ./bundles/ga4_merch_store
```

### Expected bundle output

The recipe writes to `./bundles/ga4_merch_store/`. The repository's checked-in reference bundle is at `bundles/ga4/` (same dataset, pre-generated output you can browse without re-running).

:::files
bundles/ga4/
├── index.md
├── datasets/
│   ├── index.md
│   └── ga4_obfuscated_sample_ecommerce.md
├── tables/
│   ├── index.md
│   └── events_.md
├── references/
│   ├── index.md
│   ├── joins/
│   └── metrics/
└── viz.html
:::

| Directory | Contents |
|-----------|----------|
| `datasets/` | One OKF doc for the dataset concept with overview, schema links, and example SQL |
| `tables/` | One doc for `events_` with field-level schema, metrics links, and query patterns |
| `references/` | Standalone docs minted from seeded GA4 pages (metrics definitions, join specs) |
| `index.md` | Auto-generated progressive-disclosure index at each directory level |

<ResponseExample>

```text title="Successful run (stderr)"
Enriched N concept(s) into bundles/ga4_merch_store; web pass used 2 seed(s)
```

</ResponseExample>

## Stack Overflow public dataset

The Stack Overflow recipe targets `bigquery-public-data.stackoverflow`, a mirror of the Stack Exchange Data Dump. Unlike GA4's single primary events table, this recipe exercises **multi-concept enrichment**: one community schema page often describes several tables (`posts_questions`, `posts_answers`, `users`), so a single fetched page may update multiple concept docs.

<Warning>
Stack Overflow tables are large. Keep `--web-max-pages` modest while iterating, and prefer `--concept` for smoke runs.
</Warning>

### Seed file

`okf/samples/stackoverflow/seeds.txt` points at canonical Stack Exchange schema references:

```text title="samples/stackoverflow/seeds.txt"
https://meta.stackexchange.com/questions/2677/database-schema-documentation-for-the-public-data-dump-and-sede
https://data.stackexchange.com/help
https://archive.org/details/stackexchange
```

### Enrich command

<RequestExample>

```bash title="Stack Overflow enrichment"
.venv/bin/python -m enrichment_agent enrich \
    --source bq \
    --dataset bigquery-public-data.stackoverflow \
    --web-seed-file samples/stackoverflow/seeds.txt \
    --out ./bundles/stackoverflow
```

</RequestExample>

Single-concept iteration:

```bash title="posts_questions only"
.venv/bin/python -m enrichment_agent enrich \
    --source bq \
    --dataset bigquery-public-data.stackoverflow \
    --web-seed-file samples/stackoverflow/seeds.txt \
    --concept tables/posts_questions \
    --web-max-pages 20 \
    --out ./bundles/stackoverflow
```

### Expected bundle output

:::files
bundles/stackoverflow/
├── index.md
├── datasets/
│   └── stackoverflow.md
├── tables/
│   ├── posts_questions.md
│   ├── posts_answers.md
│   ├── users.md
│   ├── votes.md
│   ├── comments.md
│   ├── badges.md
│   ├── tags.md
│   └── … (additional post-type tables)
├── references/
│   ├── sede_tables.md
│   ├── post_type_ids.md
│   ├── flag_types.md
│   └── … (enumerated types and SEDE docs)
└── viz.html
:::

| Layer | Scale in reference bundle | Role |
|-------|---------------------------|------|
| `datasets/` | 1 dataset doc | Container overview, table index, common query patterns |
| `tables/` | 17 table docs | One OKF doc per BQ table with schema and cross-links |
| `references/` | 33 reference docs | Standalone concepts minted from seeded schema and SEDE pages |

Dataset docs include frontmatter fields such as `type: BigQuery Dataset`, `resource` (BigQuery API URL), `title`, `description`, `tags`, and `timestamp`, plus markdown body sections for schema, query patterns, and citations.

## Bitcoin public dataset

The Bitcoin recipe targets `bigquery-public-data.crypto_bitcoin` — blocks, transactions, inputs, and outputs produced by the open-source `bitcoin-etl` pipeline. This recipe contrasts with GA4 (single denormalized events table) and Stack Overflow (many independent entities) by surfacing **cross-table foreign-key relationships** in prose: each `transactions` row references `blocks`, `inputs`, and `outputs`.

<Warning>
The `transactions` table is hundreds of GB. Prefer `--concept` for smoke runs and keep `--web-max-pages` low while iterating.
</Warning>

### Seed file

`okf/samples/crypto_bitcoin/seeds.txt`:

```text title="samples/crypto_bitcoin/seeds.txt"
https://github.com/blockchain-etl/bitcoin-etl
https://cloud.google.com/blog/products/gcp/bitcoin-in-bigquery-blockchain-analytics-on-public-data
```

The bitcoin-etl README documents schemas that map directly onto `crypto_bitcoin` tables. The Google Cloud blog post predates the `crypto_bitcoin` dataset name (it references the older `bitcoin_blockchain` dataset) but provides authoritative blockchain-analytics context.

### Enrich command

<RequestExample>

```bash title="Bitcoin enrichment"
.venv/bin/python -m enrichment_agent enrich \
    --source bq \
    --dataset bigquery-public-data.crypto_bitcoin \
    --web-seed-file samples/crypto_bitcoin/seeds.txt \
    --out ./bundles/crypto_bitcoin
```

</RequestExample>

Single-concept smoke run:

```bash title="transactions table only"
.venv/bin/python -m enrichment_agent enrich \
    --source bq \
    --dataset bigquery-public-data.crypto_bitcoin \
    --web-seed-file samples/crypto_bitcoin/seeds.txt \
    --concept tables/transactions \
    --web-max-pages 10 \
    --out ./bundles/crypto_bitcoin
```

### Expected bundle output

:::files
bundles/crypto_bitcoin/
├── index.md
├── datasets/
│   └── crypto_bitcoin.md
├── tables/
│   ├── blocks.md
│   ├── transactions.md
│   ├── inputs.md
│   └── outputs.md
└── viz.html
:::

The `transactions` concept doc links to sibling table docs and describes nested `inputs` and `outputs` RECORD fields, block references (`block_hash`, `block_number`, `block_timestamp`), and partitioning by `block_timestamp_month`. The dataset doc includes example SQL for block counts and transaction volume over time.

## Enrichment lifecycle

```mermaid
sequenceDiagram
    participant CLI as enrichment_agent enrich
    participant BQ as BigQuerySource
    participant BQAgent as BQ pass agent
    participant WebAgent as Web pass agent
    participant Bundle as OKF bundle directory

    CLI->>BQ: --dataset project.dataset
    CLI->>Bundle: --out ./bundles/name
    BQAgent->>BQ: List concepts (dataset + tables)
    BQAgent->>Bundle: Write one .md per concept
    Note over Bundle: Auto-generate index.md per directory
    CLI->>WebAgent: --web-seed-file seeds.txt
    WebAgent->>WebAgent: fetch_url (cap: --web-max-pages)
    WebAgent->>Bundle: Enrich existing concepts or mint references/
```

Pass behavior:

1. **BQ pass** — Writes one OKF doc per concept the source advertises, using BigQuery metadata alone.
2. **Web pass** — Receives seed URLs, fetches pages via `fetch_url`, and for each page chooses to enrich one or more existing concepts, mint a standalone `references/<slug>.md` doc, or skip. A hard `--web-max-pages` cap (default 100) and same-domain allowed-hosts filter are enforced inside the tool.

## Common CLI flags

<ParamField body="--source" type="string" required>
Source adapter. Currently only `bq` (BigQuery) is supported.
</ParamField>

<ParamField body="--dataset" type="string" required>
BigQuery identifier in `project.dataset` form.
</ParamField>

<ParamField body="--out" type="path" required>
Bundle root directory to write or update.
</ParamField>

<ParamField body="--web-seed-file" type="path">
Path to a file with one seed URL per line (`#` comments allowed). Repeatable.
</ParamField>

<ParamField body="--web-seed" type="string">
Inline seed URL. Repeatable. Alternative to `--web-seed-file`.
</ParamField>

<ParamField body="--web-allowed-host" type="string">
Extra hostname the web agent may fetch beyond seed hostnames. Repeatable.
</ParamField>

<ParamField body="--web-max-depth" type="integer" default="2">
Hard cap on hop distance from any seed URL. Seeds are depth 0.
</ParamField>

<ParamField body="--model" type="string" default="gemini-flash-latest">
Gemini model id for both BQ and web agents.
</ParamField>

<ParamField body="--billing-project" type="string">
Google Cloud project to bill for BigQuery queries. Defaults to ADC default project.
</ParamField>

## Verify output

<Steps>
<Step title="Confirm stderr summary">

After `enrich` completes, stderr reports the concept count and whether the web pass ran:

```text
Enriched 22 concept(s) into bundles/stackoverflow; web pass used 3 seed(s)
```

</Step>

<Step title="Inspect bundle structure">

Open the bundle root `index.md` and follow subdirectory links. Each concept file has YAML frontmatter (`type`, `resource`, `title`, `description`, `tags`, `timestamp`) and a markdown body with schema, query patterns, and cross-links.

</Step>

<Step title="Generate a graph viewer">

```bash
.venv/bin/python -m enrichment_agent visualize --bundle ./bundles/<name>
```

This writes `<bundle>/viz.html` — a self-contained force-directed graph of concepts, cross-links, and rendered markdown. Checked-in examples: `bundles/ga4/viz.html`, `bundles/stackoverflow/viz.html`, `bundles/crypto_bitcoin/viz.html`.

</Step>
</Steps>

<Check>
A successful recipe run produces a versionable directory tree you can commit, diff in pull requests, or hand to any OKF consumer (static file server, LLM context loader, or the bundled graph viewer).
</Check>

## Related pages

<CardGroup>
<Card title="Produce OKF bundles" href="/produce-okf-bundles">
Two-pass BQ-then-web enrichment workflow, concept scoping, and bundle directory conventions.
</Card>
<Card title="OKF enrichment CLI reference" href="/okf-enrichment-cli-reference">
Full `enrich` and `visualize` subcommand flags, environment variables, and defaults.
</Card>
<Card title="Open Knowledge Format" href="/open-knowledge-format">
OKF v0.1 bundle structure, frontmatter fields, `index.md` progressive disclosure, and cross-link semantics.
</Card>
<Card title="Visualize OKF bundles" href="/visualize-okf-bundles">
Generate and share self-contained `viz.html` graph viewers from any bundle.
</Card>
</CardGroup>