# Phoenix model configuration

> PhoenixModelConfig and PhoenixRetrievalModelConfig fields, TransformerConfig, RecsysBatch, RecsysEmbeddings, and published config.json keys.

- Repository: xai-org/x-algorithm
- GitHub: https://github.com/xai-org/x-algorithm
- Human docs: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c
- Complete Markdown: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c/llms-full.txt

## Source Files

- `phoenix/recsys_model.py`
- `phoenix/recsys_retrieval_model.py`
- `phoenix/grok.py`
- `phoenix/run_pipeline.py`
- `phoenix/runners.py`
- `phoenix/README.md`
- `phoenix/pyproject.toml`

---

---
title: "Phoenix model configuration"
description: "PhoenixModelConfig and PhoenixRetrievalModelConfig fields, TransformerConfig, RecsysBatch, RecsysEmbeddings, and published config.json keys."
---

`PhoenixModelConfig` and `PhoenixRetrievalModelConfig` are the Haiku construction records for the ranking and retrieval models. `run_pipeline.build_model_config` maps each published `config.json` into those records plus a nested `TransformerConfig` and `HashConfig`. The models do not read IDs or raw embedding tables: they consume a `RecsysBatch` of hashes and actions together with a `RecsysEmbeddings` of already-looked-up vectors.

<Note>
`phoenix/artifacts/oss-phoenix-artifacts.zip` is a Git LFS pointer until pulled. After extract, treat `retrieval/config.json` and `ranker/config.json` as the runtime source of truth. The two READMEs disagree on mini-model size: `phoenix/README.md` documents 128-dim / 4 layers / 4 heads / key 32; the root README documents 256-dim / 2 layers / 4 heads. `build_model_config` does not hardcode those sizes.
</Note>

## Construct from published artifacts

<Steps>
<Step title="Extract the artifact tree">
Unzip `phoenix/artifacts/oss-phoenix-artifacts.zip` so both `retrieval/config.json` and `ranker/config.json` sit next to their `model_params.npz` and `embedding_tables.npz`. See [Installation](/installation).
</Step>
<Step title="Load each config.json">
`run_pipeline.py` reads both files. Sequence lengths, `emb_size`, and `num_actions` used to allocate the live batch come from the **retrieval** JSON. Each model is still constructed from its own JSON.
</Step>
<Step title="Map JSON into dataclasses">
`build_model_config(config, PhoenixRetrievalModelConfig)` or `build_model_config(config, PhoenixModelConfig)` fills `HashConfig` and `TransformerConfig`, then calls `initialize()`.
</Step>
<Step title="Look up embeddings, then call make()">
Hash IDs with `build_hash_functions`, index the unified table from `build_unified_emb_table`, wrap the rows in `RecsysEmbeddings`, and call `config.make()` inside a `hk.transform` forward.
</Step>
</Steps>

## Config object graph

```mermaid
classDiagram
    class TransformerConfig {
        +emb_size int
        +key_size int
        +num_q_heads int
        +num_kv_heads int
        +num_layers int
        +widening_factor float
        +attn_output_multiplier float
        +make() Transformer
    }
    class HashConfig {
        +num_user_hashes int
        +num_item_hashes int
        +num_author_hashes int
        +num_ip_hashes int
    }
    class PhoenixModelConfig {
        +model TransformerConfig
        +emb_size int
        +num_actions int
        +history_seq_len int
        +candidate_seq_len int
        +hash_config HashConfig
        +make() PhoenixModel
    }
    class PhoenixRetrievalModelConfig {
        +model TransformerConfig
        +emb_size int
        +history_seq_len int
        +candidate_seq_len int
        +enable_linear_proj bool
        +hash_config HashConfig
        +make() PhoenixRetrievalModel
    }
    class RecsysBatch {
        +user_hashes
        +history_* hashes actions surface
        +candidate_* hashes surface
    }
    class RecsysEmbeddings {
        +user_embeddings
        +history_post_embeddings
        +candidate_post_embeddings
        +history_author_embeddings
        +candidate_author_embeddings
    }
    PhoenixModelConfig --> TransformerConfig
    PhoenixRetrievalModelConfig --> TransformerConfig
    PhoenixModelConfig --> HashConfig
    PhoenixRetrievalModelConfig --> HashConfig
    PhoenixModelConfig ..> RecsysBatch : ranks
    PhoenixRetrievalModelConfig ..> RecsysBatch : encodes user
    PhoenixModelConfig ..> RecsysEmbeddings
    PhoenixRetrievalModelConfig ..> RecsysEmbeddings
```

`TransformerConfig.make()` does not pass `emb_size` into `Transformer`. The stack width is the last dimension of the input embeddings (`[B, T, D]`), so `PhoenixModelConfig.emb_size` and `TransformerConfig.emb_size` must match the looked-up tables.

## PhoenixModelConfig

Ranking construction record. `make()` returns a `PhoenixModel` whose forward is `(RecsysBatch, RecsysEmbeddings) -> RecsysModelOutput`.

<ParamField body="model" type="TransformerConfig" required>
Nested transformer hyperparameters. Required; there is no default.
</ParamField>

<ParamField body="emb_size" type="int" required>
Embedding width `D`. Must match hash-table columns and `model.emb_size`.
</ParamField>

<ParamField body="num_actions" type="int" required>
Width of the discrete action head. Published JSON supplies this; `runners.ACTIONS` has 19 labels.
</ParamField>

<ParamField body="history_seq_len" type="int">
History positions `S`. Default `128`. Used for right-anchored RoPE when that flag is on; batch history must be padded to this length in the pipeline.
</ParamField>

<ParamField body="candidate_seq_len" type="int">
Candidate positions `C`. Default `32`. `run_pipeline.py` ranks in chunks of this length.
</ParamField>

<ParamField body="name" type="str | None">
Optional label used only in the "not initialized" warning. Default `None`.
</ParamField>

<ParamField body="fprop_dtype" type="dtype">
Forward dtype. Default `jnp.bfloat16`. `BaseModelRunner.initialize` also forces this to `bfloat16`.
</ParamField>

<ParamField body="hash_config" type="HashConfig">
Hash counts. Default `HashConfig()` (`2` user / item / author hashes, `0` IP hashes).
</ParamField>

<ParamField body="product_surface_vocab_size" type="int">
Rows in `product_surface_embedding_table`. Default `16`. Shared table for history and candidate surfaces.
</ParamField>

<ParamField body="post_age_granularity_mins" type="int">
Minutes per post-age bucket. Default `60`. Vocab size is `(4800 // granularity) + 2` (`POST_AGE_MAX_MINUTES = 4800`).
</ParamField>

<ParamField body="num_continuous_actions" type="int">
Columns of `continuous_unembeddings`. Default `8`. `runners.CONTINUOUS_ACTIONS[1]` is `dwell_time`.
</ParamField>

<ParamField body="continuous_action_hidden_dim" type="int">
Hidden width of the history-dwell MLP. Default `64`.
</ParamField>

<ParamField body="continuous_action_config" type="ContinuousActionConfig">
Normalization and unused training-loss knobs. Default `ContinuousActionConfig()` with `NormConfig(norm_scale=30.0, use_log=False)`.
</ParamField>

<ParamField body="use_ip_address" type="bool">
Stored on the config. Default `False`. The forward path does **not** read this flag; IP addition is gated by `user_ip_embeddings is not None` and `hash_config.num_ip_hashes > 0`.
</ParamField>

<ParamField body="right_anchored_rope" type="bool">
When `True`, `right_anchored_rope_positions` replaces sequential RoPE indices so the newest history token is fixed and every candidate shares position `1 + history_seq_len`. Default `False`. `build_model_config` does not set this from JSON.
</ParamField>

<ParamField body="mask_neg_feedback_on_negatives" type="bool">
Stored on the config. Default `True`. No consumer in `PhoenixModel.__call__`.
</ParamField>

`initialize()` sets the initialized flag and returns `self`. `make()` warns and initializes if that flag is still false, then builds `PhoenixModel(model=self.model.make(), config=self, fprop_dtype=self.fprop_dtype)`.

### Ranker-only nested records

| Record | Field | Default | Used at inference |
|---|---|---|---|
| `NormConfig` | `norm_scale` | `30.0` | Yes — clamps then scales dwell |
| `NormConfig` | `use_log` | `False` | Yes — `log1p` path when true |
| `ContinuousActionConfig` | `loss_weight` | `0.0` | No |
| `ContinuousActionConfig` | `loss_type` | `"mae"` | No |
| `ContinuousActionConfig` | `tweedie_power` | `1.5` | No |

`normalize_continuous_value` clips to `[0, norm_scale]`, then divides by `norm_scale` or by `log1p(norm_scale)` when `use_log` is set. History dwell is taken from `history_continuous_actions[:, :, 1]`; if that tensor is missing, the ranker feeds zeros.

### Ranker outputs

<ResponseField name="logits" type="jax.Array">
`[B, C, num_actions]` discrete engagement logits from `unembeddings` (`[emb_size, num_actions]`).
</ResponseField>

<ResponseField name="continuous_preds" type="jax.Array">
`[B, C, num_continuous_actions]` — `sigmoid(candidate_embeddings @ continuous_unembeddings)`.
</ResponseField>

Candidate tokens start at offset `1 + S`. After the transformer, `layer_norm` is applied, then only the candidate slice is projected. Isolation masking is described on [Candidate isolation](/candidate-isolation).

## PhoenixRetrievalModelConfig

Two-tower construction record. The user tower is the same `Transformer` stack as the ranker; the candidate tower is `CandidateTower`.

<ParamField body="model" type="TransformerConfig" required>
User-tower transformer. Required.
</ParamField>

<ParamField body="emb_size" type="int" required>
Shared width `D` for user pooling and the candidate tower.
</ParamField>

<ParamField body="history_seq_len" type="int">
Default `128`.
</ParamField>

<ParamField body="candidate_seq_len" type="int">
Default `32`. Sizes the dummy candidate slots `run_pipeline.py` concatenates with 64 Gaussian-noise rows so `candidate_tower_projection_*` stay in the Haiku tree.
</ParamField>

<ParamField body="fprop_dtype" type="dtype">
Default `jnp.bfloat16`.
</ParamField>

<ParamField body="hash_config" type="HashConfig">
Default `HashConfig()`.
</ParamField>

<ParamField body="product_surface_vocab_size" type="int">
Default `16`. History product-surface table only; candidates are not transformer tokens on this path.
</ParamField>

<ParamField body="enable_linear_proj" type="bool">
`CandidateTower` mode. Default `True`. `build_model_config` **forces** `True` for published checkpoints and does not read a JSON key.
</ParamField>

The retrieval transformer is called with `candidate_start_offset=None`, so it uses a standard causal mask over `[user, history]` only. User representation is the masked mean of those outputs, then L2-normalized (`EPS = 1e-12`).

`CandidateTower` when `enable_linear_proj` is true: flatten hash embeddings, SiLU MLP (`D_in → 2D → D`), L2-normalize. When false: mean over the hash axis, L2-normalize, no learned parameters.

<ResponseField name="user_representation" type="jax.Array">
`[B, D]` L2-normalized user vector.
</ResponseField>

<ResponseField name="top_k_indices" type="jax.Array">
`[B, K]` corpus indices from `jax.lax.top_k` on `user @ corpus.T`. Invalid corpus rows are set to `-1e12` when `corpus_mask` is passed.
</ResponseField>

<ResponseField name="top_k_scores" type="jax.Array">
`[B, K]` corresponding dot products.
</ResponseField>

`run_pipeline.py` does not call that `__call__` path. It uses `build_user_representation` and scores `corpus["candidate_representations"]` in NumPy.

## TransformerConfig

Defined in `phoenix/grok.py`. Ported Grok-1 decoder stack with recsys attention when `candidate_start_offset` is set.

<ParamField body="emb_size" type="int" required>
Documented width. Not forwarded into `Transformer.__init__`.
</ParamField>

<ParamField body="key_size" type="int" required>
Per-head RoPE / key dim. `model_size` defaults to `key_size * num_q_heads`.
</ParamField>

<ParamField body="num_q_heads" type="int" required>
Query heads. Must be a multiple of `num_kv_heads`.
</ParamField>

<ParamField body="num_kv_heads" type="int" required>
Key/value heads. Published JSON sets both head counts from one `num_heads` key.
</ParamField>

<ParamField body="num_layers" type="int" required>
`decoder_layer_{i}` count.
</ParamField>

<ParamField body="widening_factor" type="float">
FFN expansion. Dataclass default `4.0`. Published loader hardcodes `2.0`.
</ParamField>

<ParamField body="attn_output_multiplier" type="float">
Multiplies attention logits before a `30 * tanh(x / 30)` clip. Dataclass default `1.0`. Published loader hardcodes `0.125`.
</ParamField>

`ffn_size(emb_size, widening_factor)` is `int(widening_factor * emb_size) * 2 // 3`, then rounded up to a multiple of 8. Local demos in `run_ranker.py` / `run_retrieval.py` use `emb_size=128`, `widening_factor=2`, `key_size=64`, `num_q_heads=num_kv_heads=2`, `num_layers=2`, `attn_output_multiplier=0.125`, `history_seq_len=32`, `candidate_seq_len=8` — those are untrained demo configs, not the checkpoint.

## HashConfig

<ParamField body="num_user_hashes" type="int">
Default `2`. Width of `user_hashes` / `user_embeddings`.
</ParamField>

<ParamField body="num_item_hashes" type="int">
Default `2`. Width of post-hash tensors.
</ParamField>

<ParamField body="num_author_hashes" type="int">
Default `2`. Width of author-hash tensors.
</ParamField>

<ParamField body="num_ip_hashes" type="int">
Default `0`. Ranker `block_user_reduce` sums IP embeddings into the user token only when this is `> 0` and `user_ip_embeddings` is present.
</ParamField>

Hash `0` is padding. Reduce functions take validity from the first hash lane (`[..., 0] != 0`). Linear-congruential hashing, pad offset `65`, and the unified table are specified on [Hash embeddings](/hash-embeddings).

## RecsysBatch

`NamedTuple` of feature indices. Embeddings travel separately.

| Field | Shape | Required |
|---|---|---|
| `user_hashes` | `[B, num_user_hashes]` | yes |
| `history_post_hashes` | `[B, S, num_item_hashes]` | yes |
| `history_author_hashes` | `[B, S, num_author_hashes]` | yes |
| `history_actions` | `[B, S, num_actions]` | yes — multi-hot; retrieval infers `num_actions` from this axis |
| `history_product_surface` | `[B, S]` | yes |
| `candidate_post_hashes` | `[B, C, num_item_hashes]` | yes |
| `candidate_author_hashes` | `[B, C, num_author_hashes]` | yes |
| `candidate_product_surface` | `[B, C]` | yes |
| `history_continuous_actions` | `[B, S, num_continuous_actions]` | no — ranker dwell at index `1` |
| `candidate_impr_ts` | `[B, C]` | no — seconds; both timestamp fields required to bucket age |
| `candidate_post_creation_ts` | `[B, C]` | no |
| `user_ip_hashes` | `[B, num_ip_hashes]` | no — carried on the batch; lookup is the caller's job |

Action embeddings are `(2 * actions - 1) @ action_projection`, then zeroed where the whole action vector is false. Signed `0/1` inputs therefore become `-1/+1`.

`run_pipeline.py` sets every product-surface index to `0` and omits timestamps and continuous actions. Missing age timestamps become bucket `0` (the missing/invalid bucket). Negative age and zero timestamps also map to `0`. Overflow age (`> 4800` minutes at 60-minute granularity) is bucket `81`.

## RecsysEmbeddings

Dataclass of pre-looked-up rows. `block_*_reduce` concatenates hash lanes and projects with `proj_mat_1` (user), `proj_mat_3` (history), `proj_mat_2` (ranker candidates).

| Field | Shape |
|---|---|
| `user_embeddings` | `[B, num_user_hashes, D]` |
| `history_post_embeddings` | `[B, S, num_item_hashes, D]` |
| `candidate_post_embeddings` | `[B, C, num_item_hashes, D]` |
| `history_author_embeddings` | `[B, S, num_author_hashes, D]` |
| `candidate_author_embeddings` | `[B, C, num_author_hashes, D]` |
| `user_ip_embeddings` | optional `[B, num_ip_hashes, D]` |

The pipeline indexes a unified table: `ret_emb[user_hashes]`, `ret_emb[hist_post_h]`, and so on. Dummy candidate embeddings are zeros during retrieval user encoding.

## Published config.json keys

Both `retrieval/config.json` and `ranker/config.json` are consumed by the same helpers. Keys below are those the loader actually reads.

### Required by `build_model_config`

| Key | Maps to |
|---|---|
| `emb_size` | `Phoenix*ModelConfig.emb_size` and `TransformerConfig.emb_size` |
| `history_seq_len` | `history_seq_len` |
| `candidate_seq_len` | `candidate_seq_len` |
| `num_user_hashes` | `HashConfig.num_user_hashes` |
| `num_item_hashes` | `HashConfig.num_item_hashes` |
| `num_author_hashes` | `HashConfig.num_author_hashes` |
| `key_size` | `TransformerConfig.key_size` |
| `num_heads` | both `num_q_heads` and `num_kv_heads` |
| `num_layers` | `TransformerConfig.num_layers` |
| `num_actions` | **ranker only** — `PhoenixModelConfig.num_actions`. Retrieval still reads this key at the top of `main()` to size `history_actions`. |

### Optional JSON keys

| Key | Default if absent |
|---|---|
| `product_surface_vocab_size` | `16` |
| `post_age_granularity_mins` | `60` (ranker only) |

### Required by hashing and the unified table

| Key | Role |
|---|---|
| `user_vocab_size` | user table rows; also the item/author index offsets |
| `item_vocab_size` | item table rows |
| `author_vocab_size` | author table rows |
| `hash_params.user_hash_scales` | per-hash LCG scales |
| `hash_params.user_biases` | per-hash LCG biases |
| `hash_params.user_modulus` | LCG modulus |
| `hash_params.item_hash_scales` | item LCG scales |
| `hash_params.item_biases` | item LCG biases |
| `hash_params.item_modulus` | item modulus |
| `hash_params.author_hash_scales` | author LCG scales |
| `hash_params.author_biases` | author LCG biases |
| `hash_params.author_modulus` | author modulus |

### Hardcoded in the loader, not JSON

| Value | Where |
|---|---|
| `widening_factor=2.0` | `TransformerConfig` in `build_model_config` |
| `attn_output_multiplier=0.125` | same |
| `enable_linear_proj=True` | retrieval branch |
| pad offset `65` | `build_hash_functions` / `build_unified_emb_table` |

```text
unified table rows = 65 + user_vocab_size + item_vocab_size + author_vocab_size
         [0 .. 64]   [65 .. 65+U)   [65+U .. 65+U+I)   [65+U+I .. end)
         pad zeros   user_embeddings item_embeddings    author_embeddings
```

ID `0` stays hash `0` (padding). Non-zero IDs hash into `[1, vocab)` then add the pad/user/item offset. `embedding_tables.npz` must contain `user_embeddings`, `item_embeddings`, and `author_embeddings`.

<RequestExample>
```python
# phoenix/run_pipeline.py — published JSON → live config
kwargs = dict(
    emb_size=config["emb_size"],
    history_seq_len=config["history_seq_len"],
    candidate_seq_len=config["candidate_seq_len"],
    hash_config=HashConfig(
        num_user_hashes=config["num_user_hashes"],
        num_item_hashes=config["num_item_hashes"],
        num_author_hashes=config["num_author_hashes"],
    ),
    product_surface_vocab_size=config.get("product_surface_vocab_size", 16),
    model=TransformerConfig(
        emb_size=config["emb_size"],
        key_size=config["key_size"],
        num_q_heads=config["num_heads"],
        num_kv_heads=config["num_heads"],
        num_layers=config["num_layers"],
        widening_factor=2.0,
        attn_output_multiplier=0.125,
    ),
)
if config_class == PhoenixModelConfig:
    kwargs["num_actions"] = config["num_actions"]
    kwargs["post_age_granularity_mins"] = config.get("post_age_granularity_mins", 60)
elif config_class == PhoenixRetrievalModelConfig:
    kwargs["enable_linear_proj"] = True
mc = config_class(**kwargs)
mc.initialize()
```
</RequestExample>

### Documented mini-checkpoint table

`phoenix/README.md` lists these published mini values. Confirm against extracted JSON before depending on them.

| Parameter | Documented mini value |
|---|---|
| Embedding dimension | 128 |
| Transformer layers | 4 |
| Attention heads | 4 |
| Key size | 32 |
| Widening factor | 2 |
| History sequence length | 127 |
| Candidate sequence length | 64 |
| User / item / author vocab | 1,000,000 each |
| Hashes per entity | 2 |
| Action types | 19 |

Dataclass defaults differ (`history_seq_len=128`, `candidate_seq_len=32`). Production Phoenix is described as a larger continuously trained model; this checkout ships a frozen mini snapshot.

## Reduce projections and parameter names

These Haiku names must exist in the matching `model_params.npz` after `hk.transform`:

| Name | Owner | Shape intent |
|---|---|---|
| `proj_mat_1` | user reduce | `[num_user_hashes * D, D]` |
| `proj_mat_3` | history reduce | `[concat_width, D]` |
| `proj_mat_2` | ranker candidate reduce | `[concat_width, D]` |
| `action_projection` | both | `[num_actions, D]` |
| `product_surface_embedding_table` | both | `[product_surface_vocab_size, D]` |
| `unembeddings` | ranker | `[D, num_actions]` |
| `continuous_unembeddings` | ranker | `[D, num_continuous_actions]` |
| `history_dwell_time_proj1` / `_proj2` | ranker | `[1, hidden]` / `[hidden, D]` |
| `post_age_embedding_table` | ranker | `[post_age_vocab_size, D]` |
| `candidate_tower_projection_1` / `_2` | retrieval | `[hash_concat, 2D]` / `[2D, D]` |
| `log_temperature` | retrieval forward in `run_pipeline.py` | scalar, initialized to `0` so the published key is consumed |

History concat (ranker) is `[post hashes | author hashes | action emb | surface emb | dwell emb]`. Retrieval history omit dwell. Ranker candidates add post-age embeddings; they do not take action embeddings.

## Constraints and failures

- **Missing JSON key.** `build_model_config` / `build_hash_functions` raise `KeyError` on any required key listed above.
- **Uninitialized `make()`.** Logs `PhoenixModel {name} is not initialized. Initializing.` (or the retrieval equivalent) and continues.
- **Vocab / offset mismatch.** Hashes that land past `65 + U + I + A` index off the unified table.
- **Sequence-length split.** Live `S` and `C` come from **retrieval** JSON even when the ranker JSON differs. Rank in chunks of `candidate_seq_len`; a short last chunk is zero-padded.
- **Separate hash spaces.** Retrieval and ranker each have their own `hash_params` and embedding tables. Do not mix `ret_emb` rows with `rank_hash_*` indices.
- **Dtype.** Inference runners force `fprop_dtype = bfloat16`.
- **IP and negative-feedback flags.** `use_ip_address` and `mask_neg_feedback_on_negatives` are not wired in the published forward.
- **LFS.** Until `git lfs pull`, `config.json` is not on disk. See [Troubleshooting](/troubleshooting).

Verify a constructed config by running `uv run pytest test_recsys_model.py test_recsys_retrieval_model.py` and, after extract, `uv run run_pipeline.py --artifacts_dir artifacts/oss-phoenix-artifacts`. The ranked table is the success signal.

## Related pages

<CardGroup>
<Card title="Hash embeddings" href="/hash-embeddings">
LCG hashing, pad-offset unified tables, and RecsysBatch lookup before the transformer.
</Card>
<Card title="Candidate isolation" href="/candidate-isolation">
`make_recsys_attn_mask` rules so candidate scores stay batch-independent.
</Card>
<Card title="Run the inference pipeline" href="/run-inference-pipeline">
Load both checkpoints, encode example_sequence.json, retrieve, rank.
</Card>
<Card title="Multi-action scoring" href="/multi-action-scoring">
Per-action logits, demo weighted sums, and production weight tables.
</Card>
<Card title="Customize a user sequence" href="/customize-user-sequence">
example_sequence.json fields and history padding to history_seq_len.
</Card>
<Card title="Test Phoenix" href="/test-phoenix">
Attention-mask, retrieval, and CandidateTower assertions.
</Card>
</CardGroup>