# Run the inference pipeline

> Load retrieval and ranker checkpoints, encode example_sequence.json, retrieve from sports_corpus.npz, and rank with the published action weights.

- Repository: xai-org/x-algorithm
- GitHub: https://github.com/xai-org/x-algorithm
- Human docs: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c
- Complete Markdown: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c/llms-full.txt

## Source Files

- `phoenix/run_pipeline.py`
- `phoenix/README.md`
- `phoenix/runners.py`
- `phoenix/recsys_model.py`
- `phoenix/recsys_retrieval_model.py`
- `phoenix/run_ranker.py`
- `phoenix/run_retrieval.py`

---

---
title: "Run the inference pipeline"
description: "Load retrieval and ranker checkpoints, encode example_sequence.json, retrieve from sports_corpus.npz, and rank with the published action weights."
---

`phoenix/run_pipeline.py` is the local retrieval-then-rank entry point. It loads the exported Phoenix checkpoints, encodes a user action sequence, retrieves from the precomputed sports corpus, scores the retrieved posts with the ranker, and prints a weighted engagement table. It does not start Home Mixer, Thunder, or a Phoenix prediction cluster.

<Warning>
`phoenix/artifacts/oss-phoenix-artifacts.zip` is a Git LFS object (about 2.90 GB). If the file is a 135-byte `version https://git-lfs.github.com/spec/v1` pointer, `unzip` will fail. Pull LFS objects before extract. See [Installation](/installation) and [Troubleshooting](/troubleshooting).
</Warning>

## Prerequisites

- Python 3.11+ in `phoenix/` (`requires-python = ">=3.11"` in `phoenix/pyproject.toml`)
- Dependencies from that project: `jax==0.8.1`, `dm-haiku`, `numpy`
- Extracted artifact tree (not the LFS pointer)

<Tabs>
<Tab title="uv">

```bash
cd phoenix
uv sync
```

</Tab>
<Tab title="pip">

```bash
cd phoenix
pip install jax==0.8.1 dm-haiku numpy
```

</Tab>
</Tabs>

## Artifact layout

Extract the archive so the runner can see sibling `retrieval/`, `ranker/`, `sports_corpus.npz`, and `example_sequence.json` directories/files:

```bash
cd phoenix
unzip artifacts/oss-phoenix-artifacts.zip -d artifacts/
```

:::files
artifacts/oss-phoenix-artifacts/
  retrieval/
    model_params.npz
    embedding_tables.npz
    config.json
  ranker/
    model_params.npz
    embedding_tables.npz
    config.json
  sports_corpus.npz
  example_sequence.json
:::

The argparse default `--artifacts_dir` is `./artifacts`. After the documented unzip, pass the nested directory:

```bash
uv run run_pipeline.py --artifacts_dir artifacts/oss-phoenix-artifacts
```

| Path | Role |
| --- | --- |
| `retrieval/config.json` | Retrieval architecture, vocab sizes, `hash_params` |
| `retrieval/model_params.npz` | Haiku weights for the user tower, candidate tower, and `log_temperature` |
| `retrieval/embedding_tables.npz` | `user_embeddings`, `item_embeddings`, `author_embeddings` |
| `ranker/config.json` | Ranker architecture, `num_actions`, hash params |
| `ranker/model_params.npz` | Ranking transformer + action unembedding |
| `ranker/embedding_tables.npz` | Separate ranker hash tables |
| `sports_corpus.npz` | Precomputed candidate representations |
| `example_sequence.json` | Viewer `user_id` and history items |

Phoenix documents the sports corpus as about 537K Sports-topic posts from a six-hour window, with `example_sequence.json` as three liked/dwelled sports posts (NFL, NBA, NHL). Confirm counts from the extracted NPZ and JSON; the runner logs them at startup.

## CLI

<ParamField body="artifacts_dir" type="path" default="./artifacts">
Directory that contains `retrieval/`, `ranker/`, and the default sequence/corpus files.
</ParamField>

<ParamField body="sequence_file" type="path">
User sequence JSON. Default: `{artifacts_dir}/example_sequence.json`.
</ParamField>

<ParamField body="corpus_file" type="path">
Corpus NPZ. Default: `{artifacts_dir}/sports_corpus.npz`.
</ParamField>

<ParamField body="top_k_retrieval" type="int" default="200">
How many corpus hits to keep before ranking. Clamped to `len(post_ids)`.
</ParamField>

<ParamField body="top_k_display" type="int" default="30">
How many ranked rows to print. Clamped to the retrieved count.
</ParamField>

<CodeGroup>

```bash uv
cd phoenix
uv run run_pipeline.py \
  --artifacts_dir artifacts/oss-phoenix-artifacts \
  --top_k_retrieval 200 \
  --top_k_display 30
```

```bash python
cd phoenix
python run_pipeline.py \
  --artifacts_dir artifacts/oss-phoenix-artifacts \
  --top_k_retrieval 200 \
  --top_k_display 30
```

</CodeGroup>

Override sequence or corpus without moving files:

```bash
uv run run_pipeline.py \
  --artifacts_dir artifacts/oss-phoenix-artifacts \
  --sequence_file /path/to/sequence.json \
  --corpus_file /path/to/sports_corpus.npz
```

## Run the pipeline

<Steps>
<Step title="Confirm the LFS archive is real">
`oss-phoenix-artifacts.zip` must be a zip, not an LFS pointer. Then extract into `phoenix/artifacts/` so `artifacts/oss-phoenix-artifacts/retrieval/config.json` exists.
</Step>
<Step title="Install Phoenix deps">
From `phoenix/`, run `uv sync` (or pip-install `jax==0.8.1`, `dm-haiku`, `numpy`).
</Step>
<Step title="Invoke run_pipeline.py">
Pass `--artifacts_dir artifacts/oss-phoenix-artifacts`. Optional: `--top_k_retrieval`, `--top_k_display`, `--sequence_file`, `--corpus_file`.
</Step>
<Step title="Verify the ranked table">
Success is stdout headed `PIPELINE RESULTS — User {user_id}` with `Retrieved top K → Ranked by engagement model`, per-row scores, and a final `Weighted score range: [min, max]`. Logs also print user-repr L2 norm and retrieval score range.
</Step>
</Steps>

## What the runner does

`run_pipeline.py` does **not** use `RecsysRetrievalInferenceRunner` or `RecsysInferenceRunner`. Those classes power the random-init demos in `run_retrieval.py` / `run_ranker.py` and the unit tests. The artifact runner builds its own Haiku transforms and applies `load_model_params` checkpoints.

```mermaid
flowchart LR
  subgraph artifacts["artifacts/oss-phoenix-artifacts"]
    retFiles["retrieval/config.json<br/>model_params.npz<br/>embedding_tables.npz"]
    rankFiles["ranker/config.json<br/>model_params.npz<br/>embedding_tables.npz"]
    seq["example_sequence.json"]
    corpus["sports_corpus.npz"]
  end

  subgraph encode["Encode sequence"]
    hashFn["build_hash_functions"]
    padHist["history[:history_seq_len]<br/>right-pad with zeros"]
    lookup["unified embedding table lookup"]
  end

  subgraph retrieve["Retrieval"]
    userTower["PhoenixRetrievalModel.build_user_representation"]
    dot["corpus_repr @ user_repr"]
    topk["argpartition then argsort"]
  end

  subgraph rank["Ranking"]
    chunks["chunks of candidate_seq_len"]
    phoenix["PhoenixModel logits"]
    sig["jax.nn.sigmoid"]
    wsum["fav*1 + reply*0.5 + rt*0.3 + dwell*0.2"]
  end

  seq --> padHist
  retFiles --> hashFn
  rankFiles --> hashFn
  hashFn --> lookup
  padHist --> lookup
  lookup --> userTower
  retFiles --> userTower
  corpus --> dot
  userTower --> dot
  dot --> topk
  topk --> chunks
  rankFiles --> phoenix
  chunks --> phoenix
  phoenix --> sig
  sig --> wsum
```

### 1. Load configs and tables

The runner reads `retrieval/config.json` and `ranker/config.json`, then:

- `load_model_params` — NPZ keys are `module/path/param`, reassembled into a Haiku dict
- `load_embedding_table` — raw `user_embeddings` / `item_embeddings` / `author_embeddings`
- `build_unified_emb_table` — concatenates those three tables behind a pad of **65** rows: `[pad | user | item | author]`

`build_model_config` reconstructs `PhoenixRetrievalModelConfig` or `PhoenixModelConfig` from JSON. Retrieval always sets `enable_linear_proj=True`. Both configs hardcode transformer `widening_factor=2.0` and `attn_output_multiplier=0.125`. Ranker also reads `num_actions` and `post_age_granularity_mins` (default 60). History action-vector width comes from **`ret_cfg["num_actions"]`**.

Architecture numbers live in the extracted `config.json` files. `phoenix/README.md` documents a mini snapshot (128-d, 4 layers, 4 heads, `history_seq_len` 127, `candidate_seq_len` 64, 19 actions). The repo root README names a different snapshot (256-d, 2 layers). Trust the JSON next to the checkpoints.

### 2. Encode the user sequence

Expected JSON:

<RequestExample>

```json example_sequence.json
{
  "user_id": 123,
  "history": [
    {
      "post_id": 1987654321098765432,
      "author_id": 456,
      "actions": {
        "1": 1.0,
        "11": 1.0
      }
    }
  ]
}
```

</RequestExample>

| Field | Constraint |
| --- | --- |
| `user_id` | Integer viewer id; hashed with the user hash family |
| `history` | List; only the first `history_seq_len` items are used |
| `history[].post_id` | Placed at index `i`; unused slots stay `0` (padding) |
| `history[].author_id` | Same alignment as `post_id` |
| `history[].actions` | Map of `ActionName` index string → float; written into `history_actions[i, idx]` when `idx < num_actions` |

Hashing is the training linear-congruential map (`id * scale + bias) % modulus`), then bucketed into `[1, vocab)` with `0` reserved for pad, then offset into the unified table (`user + 65`, `item + 65 + user_vocab`, `author + 65 + user_vocab + item_vocab`). Retrieval and ranker each use their own `hash_params`.

This path always sets `history_product_surface` and `candidate_product_surface` to zeros. It does not pass `candidate_impr_ts` / `candidate_post_creation_ts`, so ranker post-age buckets are the missing/zero bucket.

Field-level sequence editing belongs on [Customize a user sequence](/customize-user-sequence). Index tables belong on [Action indices](/action-indices).

### 3. Retrieve from `sports_corpus.npz`

Required NPZ keys:

| Array | Use |
| --- | --- |
| `post_ids` | Displayed as `https://x.com/a/status/{id}` |
| `author_ids` | Ranker author hashes |
| `candidate_representations` | Precomputed L2-normalized candidate vectors `[N, D]` |
| `topics` | Optional; missing → empty strings |

The user tower is `PhoenixRetrievalModel.build_user_representation`: user + history through the transformer, masked mean pool, then L2 normalize. The forward also instantiates the candidate tower and `log_temperature` against 64 dummy “graph-negative” slots so those checkpoint parameters bind; those dummies are not scored.

Corpus search is NumPy, not `PhoenixRetrievalModel.__call__` / `jax.lax.top_k`:

```text
scores = corpus_repr @ user_repr[0]
top_idx = argpartition(scores, -K)[-K:]
top_idx = top_idx[argsort(-scores[top_idx])]
```

`K = min(--top_k_retrieval, N)`. The log line is `Retrieved K (score range: low - high)`.

### 4. Rank retrieved posts

The ranker is `PhoenixModel`: user + history + candidates, candidate-isolation attention, then unembedding to `[B, candidate_seq_len, num_actions]` logits. Retrieved hits are scored in chunks of `candidate_seq_len`. A short final chunk is zero-padded; only the real `cs` rows are kept.

```text
probs = sigmoid(logits)
weighted = fav*1.0 + reply*0.5 + retweet*0.3 + dwell*0.2
ranked  = argsort(-weighted)
```

Columns used in that sum are proto `ActionName` indices, not `runners.ACTIONS` positions 0..18:

| Constant | Index | Weight in this runner |
| --- | --- | --- |
| `IDX_FAV` (`SERVER_TWEET_FAV`) | 1 | 1.0 |
| `IDX_REPLY` (`SERVER_TWEET_REPLY`) | 4 | 0.5 |
| `IDX_RT` (`SERVER_TWEET_RETWEET`) | 6 | 0.3 |
| `IDX_DWELL` (`CLIENT_TWEET_RECAP_DWELLED`) | 11 | 0.2 |
| `IDX_VQV` (`CLIENT_TWEET_VIDEO_QUALITY_VIEW`) | 13 | printed only |
| `IDX_QUOTE` (`SERVER_TWEET_QUOTE`) | 5 | unused in the sum |

This is the **demo** combiner. Production `WeightedScorer` applies a larger signed weight table (including negative feedback), VQV eligibility, and an offset/normalization step. Do not treat the four coefficients above as Home Mixer weights. See [Multi-action scoring](/multi-action-scoring) and [Scorers and weights](/scorers-and-weights).

`RecsysInferenceRunner.rank` is a different combiner: it sigmoids logits and sorts by column **0** (`favorite_score` in `ACTIONS`). The artifact pipeline does not call it.

## Output

<ResponseExample>

```text
========================================================================================================================
PIPELINE RESULTS — User 123
History: 3 items | Corpus: 537000 posts
Retrieved top 200 → Ranked by engagement model
========================================================================================================================
Rank  Score    Ret     Fav     Reply   RT      Dwell   VQV     Topics                         Post URL
------------------------------------------------------------------------------------------------------------------------
1     0.1234   0.2100  0.0800  0.0200  0.0100  0.1500  0.0050  Sports                          https://x.com/a/status/…
…

Weighted score range: [0.0100, 0.1234]
========================================================================================================================
```

</ResponseExample>

| Column | Source |
| --- | --- |
| `Rank` | Order of `-weighted` |
| `Score` | Demo weighted sum |
| `Ret` | Retrieval dot product |
| `Fav` / `Reply` / `RT` / `Dwell` / `VQV` | `sigmoid(logits)` at the `IDX_*` columns |
| `Topics` | First 28 characters of `topics[i]` |
| `Post URL` | `https://x.com/a/status/{post_id}` |

Log lines to expect before the table: `Loading retrieval model...`, `Loading ranker model...`, `Loading corpus...` (`N posts, repr shape ...`), `User {id}, H history items`, `User repr norm=...`, `Retrieved K (score range: ...)`, `Ranking K candidates...`.

## `config.json` keys the runner reads

| Key | Used for |
| --- | --- |
| `emb_size`, `key_size`, `num_heads`, `num_layers` | `TransformerConfig` (`num_q_heads` = `num_kv_heads` = `num_heads`) |
| `history_seq_len`, `candidate_seq_len` | Sequence pad width and ranker chunk size |
| `num_user_hashes`, `num_item_hashes`, `num_author_hashes` | `HashConfig` |
| `user_vocab_size`, `item_vocab_size`, `author_vocab_size` | Unified table layout and bucket counts |
| `hash_params.{user,item,author}_{hash_scales,biases,modulus}` | Linear-congruential hashes |
| `num_actions` | History / logit width (history width from retrieval JSON) |
| `product_surface_vocab_size` | Optional; default 16 |
| `post_age_granularity_mins` | Ranker only; default 60 |

Full field semantics: [Phoenix model configuration](/phoenix-model-configuration).

## Related scripts (not this path)

| Script | What it actually runs |
| --- | --- |
| `run_pipeline.py` | Published checkpoints + sports corpus + sequence JSON |
| `run_retrieval.py` | Random-init `RecsysRetrievalInferenceRunner` + `create_example_corpus(1000)` |
| `run_ranker.py` | Random-init `RecsysInferenceRunner` + `create_example_batch`; sorts by favorite (column 0) |

`uv run pytest test_recsys_model.py test_recsys_retrieval_model.py` exercises attention masks, L2-normalized towers, and top-k shape/order on synthetic tensors. It does not load `oss-phoenix-artifacts`. See [Test Phoenix](/test-phoenix).

## Constraints and failure modes

| Symptom | Cause |
| --- | --- |
| `unzip` fails or archive is 135 bytes | LFS pointer not pulled |
| `FileNotFoundError` on `retrieval/config.json` | `--artifacts_dir` points at `artifacts/` instead of `artifacts/oss-phoenix-artifacts/` |
| Missing `sports_corpus.npz` / `example_sequence.json` | Defaults resolve under `--artifacts_dir` |
| `KeyError` on `post_ids` / `candidate_representations` / `author_ids` | Corpus NPZ is not the published sports dump |
| Haiku apply / shape errors | `config.json` does not match `model_params.npz`, or embedding table vocabs do not match hash offsets |
| Empty or all-zero history after the first `history_seq_len` items | Extra history is dropped; pad slots stay 0 |
| Ranked order disagrees with `run_ranker.py` | Different index scheme and different sort key |

This checkout does not serve production For You. Home Mixer `PhoenixScorer` calls an unpublished prediction client with a hydrated `scoring_sequence`. Local ranking here is a frozen mini checkpoint plus a four-term printout. See [Runtime boundaries](/runtime-boundaries).

## Next

<CardGroup>
<Card title="Customize a user sequence" href="/customize-user-sequence">
`example_sequence.json` fields, ActionName indices, history padding, and `--top_k_*` knobs.
</Card>
<Card title="Multi-action scoring" href="/multi-action-scoring">
Per-action logits, this runner’s weighted sum, and production weight tables.
</Card>
<Card title="Phoenix model configuration" href="/phoenix-model-configuration">
`PhoenixModelConfig`, `PhoenixRetrievalModelConfig`, `RecsysBatch`, and published `config.json` keys.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
LFS pointer failures, artifact paths, and other local run breakages.
</Card>
</CardGroup>