# Multi-action scoring > Per-action logits, demo weighted sums in run_pipeline.py, and production WeightedScorer versus RankingScorer weight tables. - Repository: xai-org/x-algorithm - GitHub: https://github.com/xai-org/x-algorithm - Human docs: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c - Complete Markdown: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c/llms-full.txt ## Source Files - `phoenix/runners.py` - `phoenix/run_pipeline.py` - `home-mixer/scorers/weighted_scorer.rs` - `home-mixer/scorers/ranking_scorer.rs` - `phoenix/recsys_model.py` - `home-mixer/scorers/phoenix_scorer.rs` --- --- title: "Multi-action scoring" description: "Per-action logits, demo weighted sums in run_pipeline.py, and production WeightedScorer versus RankingScorer weight tables." --- The Phoenix ranker decodes each candidate into a vector of raw engagement logits `[B, C, num_actions]`. Local inference turns those logits into probabilities and collapses a small subset into a display score. Home Mixer first materializes named `PhoenixScores`, then `RankingScorer` applies a much larger weight table, an offset, author-diversity decay, and an out-of-network multiplier. Numeric production weights live in unpublished `crate::params` / feature-switch types; this checkout publishes the formulas, field names, and the demo coefficients only. `PhoenixCandidatePipeline` wires `PhoenixScorer` → `RankingScorer` → `VMRanker`. `WeightedScorer`, `AuthorDiversityScorer`, and `OONScorer` remain in `home-mixer/scorers/` as compile-time or standalone variants. They are not constructed in the pipeline and are not exported from `scorers/mod.rs`. The root README still diagrams those standalone stages; the snapshot that ships here inlines diversity and OON inside `RankingScorer`. ## Scoring surfaces ```mermaid flowchart TB subgraph phoenix ["phoenix/recsys_model.py"] unemb["unembeddings [emb_size, num_actions]"] logits["logits [B, C, num_actions]"] cont["continuous_preds = sigmoid(continuous_unembeddings)"] unemb --> logits end subgraph demo ["Local demo"] pipe["run_pipeline.py: sigmoid + 4-term sum"] ranker["run_ranker.py: sigmoid, sort by p_favorite"] end subgraph mixer ["Home Mixer PhoenixCandidatePipeline"] ps["PhoenixScorer → phoenix_scores"] rs["RankingScorer → weighted_score + score"] vm["VMRanker if EnableVMRanker → score"] sel["TopKScoreSelector reads score"] ps --> rs --> vm --> sel end logits --> pipe logits --> ranker logits -.->|"production prediction service"| ps ``` | Surface | Input | Collapse | Sort / select key | |---|---|---|---| | `RecsysModel.__call__` | candidate token after the transformer | none — raw logits plus a separate continuous head | n/a | | `run_pipeline.py` | published ranker logits | `fav*1.0 + reply*0.5 + rt*0.3 + dwell*0.2` | that weighted sum | | `RecsysInferenceRunner.rank` / `run_ranker.py` | in-process ranker logits | none | `probs[:, :, 0]` (`favorite_score` in `ACTIONS` order) | | `PhoenixScorer` | remote `predict` response | none | writes `phoenix_scores` only | | `WeightedScorer` | `phoenix_scores` | compile-time `p::*` weights + offset | writes `weighted_score` only; not wired | | `RankingScorer` | `phoenix_scores` + `query.params` | feature-switch weights + offset + diversity + OON | `weighted_score` then final `score` | ## Logit head `PhoenixModelConfig.num_actions` sizes both the history action vector and the discrete head. History actions are signed (`2 * actions - 1`) and projected with `action_projection` of shape `[num_actions, emb_size]`. After the transformer and layer-norm, candidate tokens (everything from `candidate_start_offset` onward) are dotted with `unembeddings` of shape `[emb_size, num_actions]`. A second head, `continuous_unembeddings` `[emb_size, num_continuous_actions]`, produces `continuous_preds` after a sigmoid. The default `num_continuous_actions` is `8`. Discrete logits stay raw; callers apply `jax.nn.sigmoid` when they need probabilities. The published mini ranker documents **19** action types. `run_pipeline.py` reads `num_actions` from `ranker/config.json` / `retrieval/config.json`. Last dimension of `RecsysModelOutput.logits` and of `history_actions`. Sourced from the artifact `config.json` in `run_pipeline.py`; set to `len(ACTIONS)` (19) in `run_ranker.py`. Width of `continuous_preds`. Default `8`. Labels in `runners.CONTINUOUS_ACTIONS`: `reserved`, `dwell_time`, `video_watch_time`, `scroll_depth`, `reserved_3` … `reserved_6`. ## Two action-index spaces Do not mix these last-dimension conventions. | Space | Where | Favorite | Reply | Quote | Repost | Dwell | VQV | |---|---|---|---|---|---|---|---| | Proto `ActionName` | `example_sequence.json` keys, `run_pipeline.py` `IDX_*` | `1` | `4` | `5` | `6` | `11` | `13` | | Dense `ACTIONS` | `runners.py` `RankingOutput`, `run_ranker.py` | `0` | `1` | `11` | `2` | `10` | `6` | `run_pipeline.py` writes `history_actions[i, int(key)]` from the JSON `actions` map and later reads `all_probs[:, IDX_*]` with the proto indices. `RecsysInferenceRunner.hk_rank_candidates` maps `probs[:, :, 0..18]` onto `p_favorite_score` … `p_dwell_time` in `ACTIONS` order and sorts by index `0`. Negative-feedback slots in that dense layout are `14–17` (`NEGATIVE_FEEDBACK_INDICES`): not-interested, block, mute, report. Home Mixer never indexes the logit tensor. `PhoenixScorer` copies a named `PhoenixScores` struct from `predictions.candidate_scores(original_tweet_id)`. ## Local demo weighted sum After retrieval, `run_pipeline.py` ranks candidates in chunks of `candidate_seq_len`, applies `jax.nn.sigmoid` to `out.logits`, concatenates the real (unpadded) rows, then: ```python weighted = ( all_probs[:, IDX_FAV] * 1.0 + all_probs[:, IDX_REPLY] * 0.5 + all_probs[:, IDX_RT] * 0.3 + all_probs[:, IDX_DWELL] * 0.2 ) ranked = np.argsort(-weighted) ``` `IDX_QUOTE` (`5`) and `IDX_VQV` (`13`) are defined and VQV is printed, but neither enters the demo sum. There is no negative-feedback term, no offset, and no author-diversity or OON multiplier. ```bash uv run run_pipeline.py \ --artifacts_dir artifacts/oss-phoenix-artifacts \ --sequence_file artifacts/oss-phoenix-artifacts/example_sequence.json \ --top_k_retrieval 200 \ --top_k_display 30 ``` ```text PIPELINE RESULTS — User History: N items | Corpus: M posts Retrieved top 200 → Ranked by engagement model Rank Score Ret Fav Reply RT Dwell VQV Topics Post URL 1 0.12.. 0.0.. 0.0.. 0.0.. 0.0.. 0.0.. 0.0.. ... https://x.com/a/status/ Weighted score range: [low, high] ``` `Score` is the four-term sum. `Ret` is the retrieval dot product, not an engagement logit. Success is a non-empty table plus that range line. `run_ranker.py` does not use this sum. It calls `RecsysInferenceRunner.rank` on a synthetic batch and prints every `ACTIONS` probability, ordered by predicted favorite. ## Production scorer sequence `PhoenixCandidatePipeline` runs scorers in order. Each scorer must return one result per input candidate in the same order; a length mismatch becomes `Scorer length_mismatch` and the stage is skipped. ### PhoenixScorer `!query.has_cached_posts`. Cached-post requests skip inference. If `None`, every candidate gets `PostCandidate::default()` — empty `phoenix_scores`. `HomeTimelineRankedFollowing` when `query.in_network_only`, otherwise `HomeTimelineRanking`. On success, `update` copies `phoenix_scores`, `prediction_request_id`, and `last_scored_at_ms`. A failed `predict` (after optional egress fallback) returns `Err("Phoenix prediction failed: …")` for every candidate. ### RankingScorer Always enabled. For each candidate: 1. `combined = Σ apply(score_field, weight)` with missing scores as `0.0`. 2. `offset_score(combined)`. 3. `normalize_score(candidate, raw)` → stored as `weighted_score`. 4. Author-diversity multiplier on those weighted values (score-desc order, per-author occurrence count). 5. If `in_network == Some(false)`, multiply by `effective_oon_weight`. 6. Write `score` as that final value. `TopKScoreSelector` sorts on `candidate.score`, not `weighted_score`. ### VMRanker Enabled by `EnableVMRanker`. Forwards the named Phoenix fields plus `vqv_ineligible` into an unpublished ranker and may overwrite `score`. ## Weight tables Production numeric values are **not** in this checkout. `RankingScorer` reads them from `query.params`. `WeightedScorer` reads compile-time `p::*` constants from the same unpublished `params` module. | Phoenix field | RankingScorer param | WeightedScorer constant | In demo sum | |---|---|---|---| | `favorite_score` | `FavoriteWeight` | `FAVORITE_WEIGHT` | yes (`1.0` at proto index `1`) | | `reply_score` | `ReplyWeight` | `REPLY_WEIGHT` | yes (`0.5` at `4`) | | `retweet_score` | `RetweetWeight` | `RETWEET_WEIGHT` | yes (`0.3` at `6`) | | `photo_expand_score` | `PhotoExpandWeight` | `PHOTO_EXPAND_WEIGHT` | no | | `click_score` | `ClickWeight` | `CLICK_WEIGHT` | no | | `profile_click_score` | `ProfileClickWeight` | `PROFILE_CLICK_WEIGHT` | no | | `vqv_score` | `VqvWeight` (gated) | `VQV_WEIGHT` (gated) | printed only | | `share_score` | `ShareWeight` | `SHARE_WEIGHT` | no | | `share_via_dm_score` | `ShareViaDmWeight` | `SHARE_VIA_DM_WEIGHT` | no | | `share_via_copy_link_score` | `ShareViaCopyLinkWeight` | `SHARE_VIA_COPY_LINK_WEIGHT` | no | | `dwell_score` | `DwellWeight` | `DWELL_WEIGHT` | yes (`0.2` at `11`) | | `quote_score` | `QuoteWeight` | `QUOTE_WEIGHT` | no (`IDX_QUOTE` unused) | | `quoted_click_score` | `QuotedClickWeight` | `QUOTED_CLICK_WEIGHT` | no | | `quoted_vqv_score` | `QuotedVqvWeight` (gated) | — | no | | `dwell_time` | `ContDwellTimeWeight` | `CONT_DWELL_TIME_WEIGHT` | no | | `click_dwell_time` | `ContClickDwellTimeWeight` | — | no | | `follow_author_score` | `FollowAuthorWeight` | `FOLLOW_AUTHOR_WEIGHT` | no | | `not_interested_score` | `NotInterestedWeight` | `NOT_INTERESTED_WEIGHT` | no | | `block_author_score` | `BlockAuthorWeight` | `BLOCK_AUTHOR_WEIGHT` | no | | `mute_author_score` | `MuteAuthorWeight` | `MUTE_AUTHOR_WEIGHT` | no | | `report_score` | `ReportWeight` | `REPORT_WEIGHT` | no | | `not_dwelled_score` | `NotDwelledWeight` | — | no | `RankingScorer` therefore includes three fields `WeightedScorer` does not: `quoted_vqv_score`, `click_dwell_time`, `not_dwelled_score`. The local `home-mixer/candidate_pipeline/candidate.rs` `PhoenixScores` snapshot also omits those three; the production struct comes from `xai_candidate_pipeline::component_library::models::PhoenixScores`. Positive mass for `RankingScorer` is the sum of favorite, reply, retweet, photo-expand, click, profile-click, VQV, share, share-via-DM, share-via-copy-link, dwell, quote, quoted-click, quoted-VQV, and follow-author. Negative mass is `-(not_interested + block + mute + report + not_dwelled)`. `total_sum = positive_sum + negative_sum`. `WeightedScorer` uses precomputed `WEIGHTS_SUM` and `NEGATIVE_WEIGHTS_SUM` instead of summing at request time. ## Offset, VQV gates, diversity, OON Both combiners share the same offset shape (`NEGATIVE_SCORES_OFFSET` is unpublished): ```text if total_sum == 0: max(combined, 0) elif combined < 0: (combined + negative_sum) / total_sum * NEGATIVE_SCORES_OFFSET else: combined + NEGATIVE_SCORES_OFFSET ``` VQV is not a flat multiply: - `WeightedScorer`: `VQV_WEIGHT` only when `video_duration_ms > MIN_VIDEO_DURATION_MS`; otherwise `0.0`. - `RankingScorer`: `candidates_util::vqv_weight(query, candidate, MinVideoDurationMs, VqvWeight)` and `quoted_vqv_weight(..., EnableQuotedVqvDurationCheck)`. Author diversity (inlined in `RankingScorer`, also implemented standalone in `AuthorDiversityScorer`): ```text multiplier(position) = (1 - floor) * decay^position + floor ``` `RankingScorer` reads `AuthorDiversityDecay` and `AuthorDiversityFloor` from the query. Candidates are visited in descending weighted-score order; `position` is how many earlier posts from the same `author_id` were already seen. The first post from an author keeps multiplier `1.0` when `position == 0`. OON (inlined; standalone `OONScorer` only multiplies `score` by `OON_WEIGHT_FACTOR`): | Condition | Factor | |---|---| | `topic_ids` non-empty | `TopicOonWeightFactor` | | viewer age `< NewUserAgeThresholdSecs` and `followed_user_ids.len() >= NEW_USER_MIN_FOLLOWING` | `NEW_USER_OON_WEIGHT_FACTOR` | | otherwise | `OonWeightFactor` | Applied only when `in_network == Some(false)`. ## Constraints and failure modes - Scorers must preserve length and order. Drops belong in filters, not scorers. - Missing `PhoenixScores` fields contribute `0.0`. - No `scoring_sequence` → empty scores → weighted/final scores collapse toward the offset of a zero combined value. - Phoenix `predict` error → every candidate `Err`; later scorers see whatever `update` skipped. - Cached posts skip `PhoenixScorer` entirely. - Demo `weighted` and production `score` are not comparable: different index space, different terms, unpublished production weights, plus offset / diversity / OON. - `run_ranker.py` success is a favorite-ordered probability dump, not the four-term demo table. `params` (`FAVORITE_WEIGHT`, `FavoriteWeight`, `NEGATIVE_SCORES_OFFSET`, `NEW_USER_OON_WEIGHT_FACTOR`, …) is not in this tree. You cannot reproduce production `score` from the published ranker logits alone. ## Next Proto ActionName values, IDX_* constants, ACTIONS labels, and scorer field keys. Phoenix cluster and egress fallback, VM ranker, TopKScoreSelector. Load checkpoints, retrieve from sports_corpus.npz, print the ranked table. Why per-candidate logits stay batch-independent. PhoenixModelConfig, RecsysBatch, and published config.json keys including num_actions.