# Scorers and weights

> PhoenixScorer cluster and egress fallback, WeightedScorer and RankingScorer formulas, AuthorDiversityScorer decay, OON and VM rankers, TopKScoreSelector.

- Repository: xai-org/x-algorithm
- GitHub: https://github.com/xai-org/x-algorithm
- Human docs: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c
- Complete Markdown: https://grok-wiki.com/public/docs/xai-org-x-algorithm-23c09c39074c/llms-full.txt

## Source Files

- `home-mixer/scorers/mod.rs`
- `home-mixer/scorers/phoenix_scorer.rs`
- `home-mixer/scorers/weighted_scorer.rs`
- `home-mixer/scorers/ranking_scorer.rs`
- `home-mixer/scorers/author_diversity_scorer.rs`
- `home-mixer/scorers/oon_scorer.rs`
- `home-mixer/scorers/vm_ranker.rs`
- `home-mixer/selectors/top_k_score_selector.rs`

---

---
title: "Scorers and weights"
description: "PhoenixScorer cluster and egress fallback, WeightedScorer and RankingScorer formulas, AuthorDiversityScorer decay, OON and VM rankers, TopKScoreSelector."
---

`PhoenixCandidatePipeline` scores filtered `PostCandidate`s sequentially with `PhoenixScorer`, then `RankingScorer`, then `VMRanker` when `EnableVMRanker` is on. `TopKScoreSelector` sorts on `candidate.score` and keeps `TOP_K_CANDIDATES_TO_SELECT`. `ForYouCandidatePipeline` has an empty scorer list and uses `BlenderSelector` after `ScoredPostsSource` already ran this path.

Numeric defaults live in unpublished `crate::params` and feature-switch `Params`. This checkout exposes the formulas, param keys, and enable predicates, not the production weight table.

<Warning>
The root README still lists `WeightedScorer`, `AuthorDiversityScorer`, and `OONScorer` as sequential stages. `home-mixer/scorers/mod.rs` exports only `phoenix_scorer`, `ranking_scorer`, and `vm_ranker`. `RankingScorer` inlines the weighted sum, author-diversity decay, and out-of-network factor.
</Warning>

## Scoring stage

`CandidatePipeline::execute` hydrates, sources, hydrates candidates, then filters before `score()`. Enabled scorers run in vector order. Each `Scorer::run` must return one result per input candidate in the same order. A length mismatch becomes `Scorer length_mismatch expected=… got=…` for every slot; `update_all` copies only `Ok` results, so failed slots keep prior fields.

```text
filtered PostCandidate[]
        │
        ▼
 PhoenixScorer          writes phoenix_scores, prediction_request_id, last_scored_at_ms
        │               skipped when has_cached_posts
        ▼
 RankingScorer          writes weighted_score, then score (diversity + OON)
        │
        ▼
 VMRanker               overwrites score when EnableVMRanker
        │               skipped when the flag is off
        ▼
 TopKScoreSelector      sort by score desc, take TOP_K_CANDIDATES_TO_SELECT
        │
        ▼
 execute() truncate     result_size() == params::RESULT_SIZE
```

```mermaid
flowchart TB
  subgraph pipeline ["PhoenixCandidatePipeline"]
    PS["PhoenixScorer"]
    RS["RankingScorer"]
    VM["VMRanker"]
    TK["TopKScoreSelector"]
    PS --> RS --> VM --> TK
  end

  subgraph unpublished ["Unpublished in this checkout"]
    P["crate::params / feature switches"]
    U["candidates_util::vqv_weight<br/>normalize_score"]
  end

  subgraph remote ["Remote inference"]
    PC["PhoenixPredictionClient"]
    EG["EgressPhoenixPredictionClient"]
    VR["VMRankerClient"]
  end

  PS -->|"UseEgressSidecar then fallback"| EG
  PS --> PC
  RS --> P
  RS --> U
  VM --> VR
  VM --> P
  TK --> P
```

<ParamField body="has_cached_posts" type="bool">
Set by `CachedPostsQueryHydrator` when Redis returns at least `MIN_CACHED_POSTS_THRESHOLD` (`500`) posts. Disables `PhoenixScorer` (and most retrieval hydrators). `RankingScorer.enable` is always `true`, so cached candidates are re-combined from stored `phoenix_scores`.
</ParamField>

<ParamField body="scoring_sequence" type="Option<UserActionSequence>">
Filled by `ScoringSequenceQueryHydrator`. If `None`, `PhoenixScorer` returns `Ok(PostCandidate::default())` for every candidate and does not call predict.
</ParamField>

## PhoenixScorer

`PhoenixScorer` holds two `PhoenixPredictionClient`s: `phoenix_client` and `egress_client` (`EgressPhoenixPredictionClient` in prod). It is the only scorer that calls the ranker model.

### Enable and product surface

| Condition | Behavior |
|---|---|
| `query.has_cached_posts` | `enable` is `false`; no predict |
| `query.scoring_sequence.is_none()` | No RPC; defaults for every candidate |
| `query.in_network_only` | `ProductSurface::HomeTimelineRankedFollowing` |
| otherwise | `ProductSurface::HomeTimelineRanking` |

The request is `build_prediction_request(query, candidates, product_surface)` (unpublished util). Response scores are looked up by `CandidateHelpers::get_original_tweet_id()` (`retweeted_tweet_id` or `tweet_id`).

### Cluster resolution

`resolve_cluster` starts from `PhoenixInferenceClusterId`, then may replace it:

1. If `PhoenixRankerNewUserHistoryThreshold > 0` and `scoring_sequence.metadata.length` (or `0` if missing) is below that threshold, use `PhoenixRankerNewUserInferenceClusterId`.
2. Else if a decider is present:
   - `PhoenixCluster::Experiment1Fou` + `override_qf_use_lap7` → `Experiment1Lap7`
   - `PhoenixCluster::Experiment1Lap7` + `override_qf_use_fou` → `Experiment1Fou`
3. Otherwise keep the configured cluster.

### Egress fallback

<ParamField body="UseEgressSidecar" type="bool">
When true, `predict` goes to `egress_client` first. On error, logs `Egress predict failed, falling back` and retries the same `cluster` + request on `phoenix_client`. Direct Phoenix failures are not retried.
</ParamField>

A failed predict (after fallback) returns `Err("Phoenix prediction failed: …")` for every candidate. Successful predict writes:

- `phoenix_scores` from `predictions.candidate_scores`
- `prediction_request_id` = `query.prediction_id`
- `last_scored_at_ms` = `current_timestamp_millis()`

`PhoenixExperimentsSideEffect` (shadow traffic only) fans out `PhoenixCluster::VARIANTS` that report `is_shadow_eligible()` on the same egress/phoenix choice, and does not fall back.

## RankingScorer

Always enabled. Loads `ScoringWeights` from `query.params`, then for each candidate:

1. `raw = compute_weighted_score`
2. `weighted_score = normalize_score(candidate, raw)` (unpublished)
3. Author-diversity multiply on the normalized vector
4. If `in_network == Some(false)`, multiply by `effective_oon_weight`
5. Write `weighted_score` (step 2) and `score` (step 4)

`TopKScoreSelector` reads `score`, not `weighted_score`.

### Weighted sum

Missing `Option<f64>` scores contribute `0.0`. VQV weights can be zeroed by unpublished `candidates_util::vqv_weight` / `quoted_vqv_weight` using `MinVideoDurationMs`, `VqvWeight`, `QuotedVqvWeight`, and `EnableQuotedVqvDurationCheck`. `VideoDurationCandidateHydrator` fills `min_video_duration_ms`; `QuoteHydrator` fills `quoted_video_duration_ms`.

| `PhoenixScores` field | Weight param | Role in `ScoringWeights` |
|---|---|---|
| `favorite_score` | `FavoriteWeight` | positive sum |
| `reply_score` | `ReplyWeight` | positive sum |
| `retweet_score` | `RetweetWeight` | positive sum |
| `photo_expand_score` | `PhotoExpandWeight` | positive sum |
| `click_score` | `ClickWeight` | positive sum |
| `profile_click_score` | `ProfileClickWeight` | positive sum |
| `vqv_score` | `VqvWeight` (gated) | positive sum |
| `share_score` | `ShareWeight` | positive sum |
| `share_via_dm_score` | `ShareViaDmWeight` | positive sum |
| `share_via_copy_link_score` | `ShareViaCopyLinkWeight` | positive sum |
| `dwell_score` | `DwellWeight` | positive sum |
| `quote_score` | `QuoteWeight` | positive sum |
| `quoted_click_score` | `QuotedClickWeight` | positive sum |
| `quoted_vqv_score` | `QuotedVqvWeight` (gated) | positive sum |
| `follow_author_score` | `FollowAuthorWeight` | positive sum |
| `dwell_time` | `ContDwellTimeWeight` | applied, not in `total_sum` |
| `click_dwell_time` | `ContClickDwellTimeWeight` | applied, not in `total_sum` |
| `not_interested_score` | `NotInterestedWeight` | negative sum |
| `block_author_score` | `BlockAuthorWeight` | negative sum |
| `mute_author_score` | `MuteAuthorWeight` | negative sum |
| `report_score` | `ReportWeight` | negative sum |
| `not_dwelled_score` | `NotDwelledWeight` | negative sum |

```text
positive_sum = favorite + reply + retweet + photo_expand + click + profile_click
             + vqv + share + share_via_dm + share_via_copy_link
             + dwell + quote + quoted_click + quoted_vqv + follow_author

negative_sum = -(not_interested + block_author + mute_author + report + not_dwelled)
total_sum    = positive_sum + negative_sum

offset(combined):
  total_sum == 0        → combined.max(0)
  combined < 0          → (combined + negative_sum) / total_sum * NEGATIVE_SCORES_OFFSET
  else                  → combined + NEGATIVE_SCORES_OFFSET
```

`NEGATIVE_SCORES_OFFSET` is an unpublished crate constant, not a per-request param.

### Author diversity

After normalization, candidates are sorted by weighted score descending. Per `author_id`, the *n*th appearance (`n` starting at 0) is multiplied by:

```text
(1 - AuthorDiversityFloor) * AuthorDiversityDecay^n + AuthorDiversityFloor
```

The first post from an author keeps a multiplier of `1.0`. Later posts decay toward `AuthorDiversityFloor`.

### Out-of-network factor

Applied only when `candidate.in_network == Some(false)` (`None` and `Some(true)` are unchanged):

| Query state | Factor |
|---|---|
| `topic_ids` non-empty | `TopicOonWeightFactor` |
| user snowflake age `< NewUserAgeThresholdSecs` and `followed_user_ids.len() >= NEW_USER_MIN_FOLLOWING` | `NEW_USER_OON_WEIGHT_FACTOR` |
| otherwise | `OonWeightFactor` |

Age uses unpublished `duration_since_creation_opt(query.user_id)`. `NEW_USER_MIN_FOLLOWING` and `NEW_USER_OON_WEIGHT_FACTOR` are crate constants.

## Unwired files in `home-mixer/scorers/`

`weighted_scorer.rs`, `author_diversity_scorer.rs`, and `oon_scorer.rs` are not modules in `scorers/mod.rs` and are not boxed into `PhoenixCandidatePipeline`. They import the older `crate::candidate_pipeline::candidate` types (that module also is not compiled). Treat them as a snapshot of the decomposition `RankingScorer` now owns.

### WeightedScorer

Writes only `weighted_score`. Same `apply(score, weight)` pattern with crate constants (`FAVORITE_WEIGHT`, …) instead of feature switches. Differences from `RankingScorer`:

- No `quoted_vqv_score`, `click_dwell_time`, or `not_dwelled_score`
- VQV weight is `VQV_WEIGHT` only when `video_duration_ms > MIN_VIDEO_DURATION_MS`, else `0.0`
- Offset uses `WEIGHTS_SUM` / `NEGATIVE_WEIGHTS_SUM` constants

### AuthorDiversityScorer

Constructed with `AUTHOR_DIVERSITY_DECAY` and `AUTHOR_DIVERSITY_FLOOR`. Sorts by `weighted_score` (missing sorts as `-∞`) and writes `score = weighted_score * multiplier` using the same `(1 - floor) * decay^position + floor` formula. Does not apply an OON factor.

### OONScorer

`score = base_score * OON_WEIGHT_FACTOR` when `in_network == Some(false)`; otherwise leaves `score` unchanged. No topic or new-user branch.

## VMRanker

<ParamField body="EnableVMRanker" type="bool" required>
`VMRanker.enable`. When false, `RankingScorer`'s `score` is what `TopKScoreSelector` sees.
</ParamField>

On enable, `VMRankerCluster::parse(VMRankerClusterId)` and `client.rank` run. gRPC failure returns `Err("VMRanker gRPC call failed: …")` for every candidate. On success, `score` is the response value for `tweet_id`, or the previous `c.score` if that id is missing.

`RankRequest` includes:

| Field | Source |
|---|---|
| `viewer_id` | `query.user_id` |
| `request_timestamp_ms` | `query.request_time_ms` |
| `value_model_id` | `VMRankerValueModelId` |
| `viewer_following_count` | `followed_user_ids.len()` |
| `new_user_age_threshold_secs` | `NewUserAgeThresholdSecs` |
| `dpp_params` | `Some` only if `VMRankerDppTheta > 0` or `VMRankerDppMaxSelectedRank > 0` |

Each `RankCandidate` copies Phoenix action scores (including `not_dwelled_score`, `dwell_time`, `click_dwell_time`; not `quoted_vqv_score`), plus `in_network` (default `false`), `is_retweet` / `is_reply`, follower count, and `vqv_ineligible` when `vqv_weight == 0.0`.

## TopKScoreSelector

```rust
fn score(&self, candidate: &PostCandidate) -> f64 {
    candidate.score.unwrap_or(f64::NEG_INFINITY)
}
fn size(&self) -> Option<usize> {
    Some(params::TOP_K_CANDIDATES_TO_SELECT)
}
```

`Selector::select` sorts descending, then `split_off` at `size`. Missing `score` sorts last. After selection, `execute` still truncates the selected list to `PhoenixCandidatePipeline::result_size()` (`params::RESULT_SIZE`) before side effects.

## Local demo weights

`phoenix/run_pipeline.py` is the only runnable weighted sum in this checkout. After sigmoid on ranker logits it uses a four-term demo mix, not `RankingScorer`:

```python
weighted = (
    all_probs[:, IDX_FAV] * 1.0
    + all_probs[:, IDX_REPLY] * 0.5
    + all_probs[:, IDX_RT] * 0.3
    + all_probs[:, IDX_DWELL] * 0.2
)
```

`IDX_FAV = 1`, `IDX_REPLY = 4`, `IDX_RT = 6`, `IDX_DWELL = 11`. `IDX_VQV = 13` is printed, not mixed. Production `runners.ACTIONS` labels (`favorite_score`, `reply_score`, `repost_score`, …) align with Phoenix score keys; Home Mixer uses `retweet_score` rather than `repost_score`.

## Failure and skip matrix

| Event | Candidates after the scorer |
|---|---|
| `PhoenixScorer` disabled (`has_cached_posts`) | Prior `phoenix_scores` unchanged |
| Missing `scoring_sequence` | Defaults; `RankingScorer` still runs |
| Egress predict fails, sidecar on | Retry on `phoenix_client` |
| Phoenix predict fails | All `Err`; no `phoenix_scores` update |
| Scorer length mismatch | All `Err`; prior fields kept |
| `EnableVMRanker` false | `RankingScorer.score` used for TopK |
| VM gRPC error | All `Err`; `RankingScorer.score` kept |
| VM response missing a `tweet_id` | Keep existing `score` |

<Note>
`crate::params`, `crate::util::score_normalizer`, `crate::util::candidates_util`, `PhoenixPredictionClient`, and `PhoenixScores` (re-exported from `xai_candidate_pipeline`) are not buildable from this snapshot. Local verification of the *formulas* is `run_pipeline.py`; verification of Home Mixer scoring requires the unpublished crates.
</Note>

## Next

<CardGroup>
  <Card title="Multi-action scoring" href="/multi-action-scoring">
    Per-action logits, the demo weighted sum, and how production weight tables differ.
  </Card>
  <Card title="Action indices" href="/action-indices">
    `ActionName` indices, `IDX_*` constants, `runners.ACTIONS`, and scorer weight keys.
  </Card>
  <Card title="For You request lifecycle" href="/request-lifecycle">
    Where the scorer vector sits in `CandidatePipeline.execute`.
  </Card>
  <Card title="Add a pipeline component" href="/add-pipeline-component">
    `Scorer` length-match and `update` contracts.
  </Card>
  <Card title="Runtime boundaries" href="/runtime-boundaries">
    What this checkout can run versus unpublished Home Mixer params and clients.
  </Card>
  <Card title="In-network and out-of-network" href="/in-network-out-of-network">
    How `in_network` and `in_network_only` change sourcing and the OON factor.
  </Card>
</CardGroup>