# catalog.yaml manifest reference

> scope, snapshot, publishing, reference, aliases, entry and aspect types, entryLinks reconciliation rules, and layout selection for YAML versus Markdown knowledge-base mode.

- Repository: GoogleCloudPlatform/knowledge-catalog
- GitHub: https://github.com/GoogleCloudPlatform/knowledge-catalog
- Human docs: https://grok-wiki.com/public/docs/googlecloudplatform-knowledge-catalog-9cee6ee3cba5
- Complete Markdown: https://grok-wiki.com/public/docs/googlecloudplatform-knowledge-catalog-9cee6ee3cba5/llms-full.txt

## Source Files

- `agents/mdcode/README.md`
- `agents/mdcode/src/libts/manifest.ts`
- `agents/mdcode/docs/concept.md`
- `agents/mdcode/docs/design.md`
- `toolbox/mdcode/docs/spec.md`
- `agents/mdcode/src/libts/layout.ts`

---

---
title: catalog.yaml manifest reference
description: scope, snapshot, publishing, reference, aliases, entry and aspect types, entryLinks reconciliation rules, and layout selection for YAML versus Markdown knowledge-base mode.
---

The `catalog.yaml` manifest is the configuration contract for a kcmd Metadata as Code workspace. It declares which Knowledge Catalog resources the workspace owns, which entry and aspect types to pull locally, which subsets to publish back, and optional read-only reference layers for grounding. `kcmd init` writes a minimal manifest containing only `scope`; you extend `snapshot`, `publishing`, and `reference` before running `pull`, `reference`, or `push`.

## Manifest location

The manifest lives at the workspace root:

```text
my-workspace/
├── catalog.yaml          # Sync configuration
└── catalog/              # Pulled and edited metadata artifacts
```

`CatalogManifest.load()` validates the file with a Zod schema, resolves the scope into a `CatalogSource`, and selects the on-disk layout from that source type.

## Top-level structure

| Field | Required | Purpose |
| --- | --- | --- |
| `scope` | Yes | Primary catalog resource(s) this workspace manages |
| `resourceAlias` | No | Custom aliases for aspect, glossary, or entryLink types |
| `snapshot` | No | Entry, aspect, and entryLink types to pull into `catalog/` |
| `publishing` | No | Subset of snapshot types written back on `push` |
| `reference` | No | Read-only scope and snapshot for `*.ref.yaml` grounding layers |
| `entryLinkTypes` | No | Optional top-level list; persisted by `save()` but link sync is driven by `snapshot.entryLinks` and `publishing.entryLinks` |

<RequestExample>

```yaml title="Full manifest example"
scope: bq-dataset.my-project-id.my-dataset-id

resourceAlias:
  guidelines:
    aspect: data-agents-project.global.guidelines

snapshot:
  entries:
    - dataplex-types.global.bigquery-table
  aspects:
    - dataplex-types.global.schema
    - dataplex-types.global.bigquery-table
    - dataplex-types.global.storage
    - dataplex-types.global.overview
    - guidelines
  entryLinks:
    - definition
    - synonym

publishing:
  aspects:
    - dataplex-types.global.overview
    - guidelines
  entryLinks:
    - definition

reference:
  scope: bq-dataset.my-project-id.my-dataset-id
  snapshot:
    entries:
      - dataplex-types.global.bigquery-table
    aspects:
      - dataplex-types.global.schema
      - dataplex-types.global.overview
    entryLinks:
      - definition
      - synonym
```

</RequestExample>

## scope

<ParamField body="scope" type="string | string[]" required>
Unified identifier for the workspace source of truth. Format: `<type>.<name>`.

For BigQuery multi-dataset workspaces, `scope` may be a YAML array of `bq-dataset.<project>.<dataset>` strings. Array scopes support only `bq-dataset`; mixing scope types in one array is rejected.
</ParamField>

### Supported scope types

| Scope prefix | `kcmd init` flag | Name format | On-disk layout |
| --- | --- | --- | --- |
| `bq-dataset` | `--bigquery-dataset` | `<project>.<dataset>` (repeat flag for multiple) | YAML (Standard) |
| `kb` | `--kb` | `<project>.<location>.<entry-group-id>` | Markdown (Documents) |
| `entryGroup` | `--entry-group` | `<project>.<location>.<entry-group-id>` | YAML (Standard) |
| `biglake-namespace` / `biglake-iceberg-namespace` | `--biglake-namespace --iceberg` | `<project>.<catalog>.<namespace>` | YAML (Standard) |
| `glossary` | `--glossary` | `<project>.<location>` or `<project>.<location>.<glossary-id>` (comma-separated IDs or display names) | YAML (Standard) |

The scope type determines:

- Which GCP APIs `kcmd` calls during `pull` and `push`
- The directory hierarchy under `catalog/`
- Whether entries are **ingested** (system-managed, e.g. BigQuery tables) or user-managed (Knowledge Base pages, custom entries)

<AccordionGroup>
<Accordion title="BigQuery multi-dataset scope">

```yaml
scope:
  - bq-dataset.my-project.dataset-a
  - bq-dataset.my-project.dataset-b
```

`kcmd` joins the datasets internally and emits one scope entry per dataset when saving the manifest.

</Accordion>
<Accordion title="Glossary location mode">

```yaml
scope: glossary.my-project.us-central1
```

When no glossary ID is provided after `<project>.<location>`, the workspace operates in location mode and resolves glossaries at pull time.

</Accordion>
</AccordionGroup>

## snapshot

The `snapshot` block defines what metadata `kcmd pull` downloads into editable local files.

<ParamField body="snapshot.entries" type="string[]">
Entry types to include. Each value must be a three-part type reference: `<project>.<location>.<type-id>` (for example `dataplex-types.global.bigquery-table`). Required aspects for listed entry types are fetched implicitly during type registration.
</ParamField>

<ParamField body="snapshot.aspects" type="string[]">
Aspect types to pull. Values may be fully qualified three-part references or short aliases such as `overview` or `schema`. Aliases resolve through built-in defaults and any custom `resourceAlias` mappings.
</ParamField>

<ParamField body="snapshot.entryLinks" type="string[]">
EntryLink types to fetch during `pull`. When present and non-empty, `kcmd` calls `lookupEntryLinks` for every pulled entry and inlines results into entry artifacts. Omit or leave empty to skip link fetching entirely.
</ParamField>

### How pulled entryLinks are stored

| Link source path | Local placement |
| --- | --- |
| Entry-level (no `Schema.<field>` path) | Top-level `links.<type>[]` on the entry YAML or Markdown frontmatter |
| Column-level (`Schema.<field>` path) | `aspects.schema.fields[].links.<type>[]` |

Glossary term targets are written in human-readable form (`<project>.<location>.<glossary-display>.<term-display>`) with the full UID resource path preserved in each link's `id` field for round-trip `push`.

## publishing

The `publishing` block declares which local edits `kcmd push` writes back to Knowledge Catalog. **Publishing must be a subset of snapshot**: aspect, entry, and entryLink types listed in `publishing` must also appear in the corresponding `snapshot` lists, or manifest loading fails.

<ParamField body="publishing.entries" type="string[]">
Entry types eligible for publish-side create/delete semantics. Must be listed in `snapshot.entries`.
</ParamField>

<ParamField body="publishing.aspects" type="string[]">
Aspect types written on `push`. Aspects not listed here are kept locally for context but not uploaded. For ingested entries (BigQuery), required aspects are never pushed even if listed.
</ParamField>

<ParamField body="publishing.entryLinks" type="string[]">
EntryLink types reconciled on `push`. Must be listed in `snapshot.entryLinks`. Omit the field or leave it empty to avoid taking responsibility for link mutations — useful for read-only link workspaces.
</ParamField>

<ResponseExample>

```yaml title="Enrichment-focused publishing"
snapshot:
  entries:
    - dataplex-types.global.bigquery-table
  aspects:
    - overview
    - descriptions
    - queries
    - schema          # local context only
  entryLinks:
    - definition

publishing:
  aspects:
    - overview
    - descriptions
    - queries
  entryLinks:
    - definition
```

</ResponseExample>

Files ending in `.ref.yaml` are always skipped during `push`.

## reference

The optional `reference` block configures read-only grounding layers fetched by `kcmd reference` (not `pull`). Reference artifacts are saved as sibling `*.ref.yaml` files and are never pushed.

<ParamField body="reference.scope" type="string | string[]" required>
Source scope for reference data. Uses the same `<type>.<name>` format as the primary `scope`. Commonly points at the same BigQuery dataset or a glossary used for enrichment grounding.
</ParamField>

<ParamField body="reference.snapshot" type="object">
Same shape as the primary `snapshot` block: `entries`, `aspects`, and optional `entryLinks`. When `reference.snapshot.entryLinks` is set, `.ref.yaml` baselines include pre-edit link state so diffs against live `.yaml` files surface only enrichment changes.
</ParamField>

<Steps>
<Step title="Pull editable metadata">

Run `kcmd pull` to populate `catalog/` from the primary `scope` and `snapshot` configuration.

</Step>
<Step title="Pull reference baselines">

Run `kcmd reference` to fetch `*.ref.yaml` siblings from `reference.scope` and `reference.snapshot`.

</Step>
<Step title="Edit and push">

Modify editable files only. `push` uploads aspects and entryLinks declared in `publishing`, ignoring `.ref.yaml`.

</Step>
</Steps>

## resourceAlias

<ParamField body="resourceAlias" type="record<string, record<string, string>>">
Maps a short alias to exactly one resource. Each alias entry must contain a single key among `aspect`, `glossary`, or `entryLink`.

```yaml
resourceAlias:
  guidelines:
    aspect: data-agents-project.global.guidelines
  ecommerce:
    glossary: data-gov-project.global.ecommerce-glossary
```

Custom aliases cannot override built-in defaults. Duplicate aliases or duplicate resource mappings raise manifest parse errors.
</ParamField>

### Built-in aliases

These resolve without declaring `resourceAlias`:

| Alias | Resolves to | Resource kind |
| --- | --- | --- |
| `bigquery-dataset` | `dataplex-types.global.bigquery-dataset` | aspect |
| `bigquery-table` | `dataplex-types.global.bigquery-table` | aspect |
| `schema` | `dataplex-types.global.schema` | aspect |
| `storage` | `dataplex-types.global.storage` | aspect |
| `overview` | `dataplex-types.global.overview` | aspect |
| `definition` | `dataplex-types.global.definition` | entryLink |
| `synonym` | `dataplex-types.global.synonym` | entryLink |
| `related` | `dataplex-types.global.related` | entryLink |
| `schema-join` | `dataplex-types.global.schema-join` | entryLink |

Use project IDs, not project numbers, when authoring metadata references.

## Entry and aspect type references

All manifest type lists use Dataplex three-part references: `<project>.<location>.<resource-id>`.

| Manifest list | Alias support | Validation |
| --- | --- | --- |
| `snapshot.entries` | No — use fully qualified names | Must split into exactly three dot-separated parts |
| `snapshot.aspects` | Yes | Resolved alias must be three-part |
| `snapshot.entryLinks` | Yes | Resolved alias must be three-part |
| `publishing.*` | Same as snapshot | Must be subset of corresponding snapshot list |

During workspace initialization, `kcmd` fetches type definitions from the Catalog API and registers required aspects for listed entry types automatically.

## entryLinks reconciliation

When `publishing.entryLinks` is declared, `push` reconciles local vs remote `EntryLink` resources per entry for those types only.

### Reconciliation algorithm

1. **Serialize local links** — `toServiceEntryLinks` reads top-level `links` and `aspects.schema.fields[].links`, filtering to types in `publishing.entryLinks`.
2. **Fetch remote links** — `lookupEntryLinks` queries the catalog for the same link types.
3. **Normalize both sides** — Comparison keys use an unwrap-and-normalize strategy:
   - Strip `@dataplex` proxy shells from target references
   - Canonicalize project segments to project ID (not number)
   - Include source path (for example `Schema.customer_id`) so column-level links stay distinct
4. **Keep matches** — Links with equal keys are preserved; no delete-and-recreate cycle.
5. **Delete orphans** — Remote links of configured types with no local match are deleted.
6. **Create new** — Local links with no remote match are created with generated link IDs.

<AccordionGroup>
<Accordion title="Comparison key format">

Each link is keyed as:

```text
<normalized-link-type>|<normalized-target>|<source-path>
```

The `source-path` segment is empty for entry-level links.

</Accordion>
<Accordion title="Glossary target resolution on push">

Local link `target` values may be human-readable glossary references. When `id` is set (populated on `pull`), `push` prefers `id` to reconstruct the exact catalog UID. Otherwise it resolves through the workspace `scope` or `reference.scope` via `serviceName()`.

</Accordion>
<Accordion title="EntryLink aspect updates">

EntryLink updates when only link aspects differ are not yet implemented (`TODO` in sync). Matching links are kept as-is.

</Accordion>
</AccordionGroup>

### Local entryLink artifact shape

```yaml
aspects:
  schema:
    fields:
      - name: customer_id
        dataType: STRING
        links:
          definition:
            - target: my-project.global.business-glossary.customer-id
              id: projects/my-project/locations/global/glossaries/biz/terms/customer-id

links:
  related:
    - target: my-other-project.us.docs-eg.runbook-page
```

## Layout selection

Layout is **not** a manifest field. `CatalogSnapshot` selects the layout from `manifest.source.layout`, which each scope type sets at initialization:

| Layout | Scope types | Entry file pattern | Unstructured text |
| --- | --- | --- | --- |
| **Standard** (YAML) | `bq-dataset`, `entryGroup`, `biglake-*`, `glossary` | `<entry-id>.yaml` plus optional `<entry-id>.<aspect>.md` sidecars | Sidecar Markdown files |
| **Documents** (Markdown) | `kb` | `<entry-id>.md` | YAML frontmatter + Markdown body (`overview.content`) |

```text
Standard layout (BigQuery)
├── catalog.yaml
└── catalog/bigquery/<project>/<dataset>/
    ├── orders.yaml
    ├── orders.ref.yaml
    └── orders.overview.md

Documents layout (Knowledge Base)
├── catalog.yaml
└── catalog/<namespace>/<project>/<location>/
    ├── page1.md
    └── playbooks/mbr.md
```

Knowledge Base entries combine metadata and content in one file:

```markdown
---
type: dataplex-types.global.entry
title: My Page Title
catalogEntry:
  id: my-page-id
---
# Page body

This Markdown body maps to the `overview.content` aspect.
```

## Validation errors

Manifest loading fails fast on structural problems:

| Error | Cause |
| --- | --- |
| `scope '<value>' is invalid` | Missing dot separator in scope string |
| `scope array cannot be empty` | Empty `scope: []` |
| `Unsupported scope type in multiple scopes` | Non-`bq-dataset` entry in scope array |
| `Invalid Entry/Aspect/EntryLink Type` | Type reference does not resolve to three parts |
| `Publishing ... is not listed in snapshot` | `publishing` entry, aspect, or entryLink not in `snapshot` |
| `Alias ... has multiple mappings` | `resourceAlias` entry contains more than one resource key |
| `Cannot define predefined alias` | Custom alias collides with a built-in default |

## Initialization vs full configuration

`kcmd init` writes a scope-only manifest:

<ResponseExample>

```yaml title="After kcmd init --bigquery-dataset my-project.my-dataset"
scope: bq-dataset.my-project.my-dataset
```

</ResponseExample>

Add `snapshot`, `publishing`, and optional `reference` before the first `pull`. For enrichment workflows, declare every aspect and link type agents may read in `snapshot`, then restrict `publishing` to the types humans or CI should deploy.

<Tabs>
<Tab title="BigQuery dataset">

```yaml
scope: bq-dataset.ecommerce-prod.ecommerce-dataset
snapshot:
  entries:
    - dataplex-types.global.bigquery-dataset
    - dataplex-types.global.bigquery-table
  aspects:
    - overview
    - schema
    - storage
  entryLinks:
    - definition
publishing:
  aspects:
    - overview
  entryLinks:
    - definition
```

</Tab>
<Tab title="Knowledge Base">

```yaml
scope: kb.ecommerce-prod.global.mbr-kb
snapshot:
  entries:
    - dataplex-types.global.document
  aspects:
    - overview
publishing:
  aspects:
    - overview
```

</Tab>
<Tab title="With reference layer">

```yaml
scope: bq-dataset.my-project.my-dataset
snapshot:
  entries:
    - dataplex-types.global.bigquery-table
  aspects:
    - overview
    - schema
  entryLinks:
    - definition
publishing:
  aspects:
    - overview
  entryLinks:
    - definition
reference:
  scope: bq-dataset.my-project.my-dataset
  snapshot:
    aspects:
      - schema
    entryLinks:
      - definition
```

</Tab>
</Tabs>

## Related pages

<CardGroup cols={2}>
<Card title="Metadata as Code" href="/metadata-as-code">
kcmd workspace model, pull/push sync, reference layers, and glossary scope overview.
</Card>
<Card title="Sync catalog metadata" href="/sync-catalog-metadata">
Initialize workspaces per source type, pull snapshots, check status, and push edits.
</Card>
<Card title="kcmd CLI reference" href="/kcmd-cli-reference">
`init`, `pull`, `push`, `reference`, and authentication flags.
</Card>
<Card title="Publish enriched metadata" href="/publish-enriched-metadata">
Push mdcode workspaces after enrichment without modifying reference layers.
</Card>
</CardGroup>