# Ranking and Fusion

Search Engineering · Course · Part 1 · Chapter 5

Interactive edition: https://mutapa.dev/search-engineering/course/ranking-and-fusion

Ranking is the staged allocation of expensive attention across more candidates than any model can inspect, moving from cheap wide passes to costly narrow ones.

[Chapter 4](https://mutapa.dev/search-engineering/course/retrieval.md) ended with a fused candidate set. For the request “a space movie where the dad ages slowly”, dense retrieval recovered *Interstellar*, and fusion kept it among the leading candidates. That is enough for retrieval to succeed, but the user does not inspect the candidate set. The user sees an ordered list, and a relevant film placed below weaker alternatives may still go unnoticed. Ranking begins where retrieval stops: with plausible documents in hand and a decision about which should appear first.

That decision can use evidence the retrievers did not consider. Text scores describe lexical or semantic fit, while behavior, availability, recency, popularity, price, and distance may determine whether a result can satisfy the request now. These signals also have different computational costs. A stored popularity value is cheap to read; a model that examines the query and document together is far more expensive. Applying every signal and model to every document would spend most of the latency budget on candidates that inexpensive evidence could have removed.

A production system manages that tension through a pipeline of phases ordered by cost. Cheap scorers search broadly, intermediate phases combine domain and behavioral evidence, and expensive rerankers inspect only the survivors. Each phase concentrates more computation on fewer candidates, but every reduction also creates a recall ceiling: a document removed early cannot be restored later. The central design problem is to spend enough computation to improve the order without narrowing the candidate set before the relevant evidence has a chance to be judged.

## Set the candidate budget

A cost-ordered pipeline becomes operational when every phase has a *candidate window*, the maximum number of documents it can pass to the next scorer. One request path might retrieve 1,000 candidates, compute richer features for 100, rerank 20 with a cross-encoder, and show five results or send five passages to a generator. These counts turn an abstract sequence of models into an explicit allocation of computation.

Each window sets both a latency budget and a recall limit. A larger window gives the next phase more opportunities to recover a relevant document, but its cost usually grows with every query-document pair it must score. A smaller window saves time by making some mistakes permanent. If the relevant document enters at rank 150 and the next scorer receives only 100 candidates, changing that scorer cannot repair the result because the document never reaches it.

The window depths should therefore come from labeled queries rather than copied defaults. Record where relevant documents appear before each phase, then choose a depth that covers the required share of that rank distribution within the latency target. Measure what the phase changes near the boundary as well. If relevant documents regularly sit just outside the window, widen it or improve the preceding phase. If an expensive scorer rarely changes the order of its lowest candidates, narrowing the window may recover latency without reducing quality. The budget is justified only when the additional candidates create a measurable opportunity to improve the final list.

## Add behavioral and domain features

Retrieval scores explain how well a document matches the request, but a match can still miss the action the user intended. A retailer searching for “iPad,” for example, may see chargers above tablets because accessory descriptions repeat the product name and receive strong BM25 scores. The text evidence is real; it is simply incomplete. When many users skip those chargers, open the tablets, and complete purchases, their behavior supplies evidence that the lexical score could not observe.

*Signals boosting* turns those events into a query-document feature. An offline aggregation groups events by a normalized query and document, assigns each event a weight, and stores the total for a cheap lookup during ranking. A starting scheme might assign a click 1, an add-to-cart 10, and a purchase 25, while skips or returns contribute negative values. Those ratios express assumptions about how strongly each action indicates success, so they must be checked against the product's actual user journey rather than treated as universal constants.

The aggregation also needs controls for identity and time. Query normalization lets “iPad” and “ipad” share evidence, while per-user deduplication prevents one person or account from dominating the total. Time decay reduces an event's contribution according to $w \cdot 0.5^{\,t/h}$, where `w` is its original weight, `t` is its age, and `h` is the chosen half-life. A news index may need a half-life measured in hours, while a job board may use weeks. The correct interval follows how quickly relevance changes in that domain.

Behavior is only one source of inexpensive ranking evidence. Availability, recency, popularity, price, and distance can be read from the document or computed from the request, including for items with no interaction history. These domain features express constraints that text similarity cannot recover. An unavailable tablet should not rank first because its description matches perfectly, and a nearby restaurant may be more useful than a slightly closer semantic match across the city.

The two feature families cover different parts of the traffic. Behavioral boosts can memorize successful choices for frequent queries over stable documents, but they remain sparse for rare queries, new items, and rapidly changing catalogs. Domain features still apply in those cases, although deciding how much each should matter requires more than a fixed additive boost. Later sections turn these features into inputs to a learned ranker and examine how click-derived evidence must be corrected before it becomes a training label.

Categorical request details need an explicit role. A hard requirement such as language, tenant, availability, or permission should filter candidates before ranking. A preference such as genre or locality may justify a request-conditioned boost when mismatches remain eligible. A category can instead enter a learned ranker as a feature when its effect depends on other evidence. For each category, record its source, allowed values, default for missing data, and the query classes where it is expected to help. Validate the resulting ordering on those slices so a broad average cannot hide damage to a minority category.

## Apply cross-encoder reranking

A hybrid retriever compares the query and each document through representations computed separately. A *cross-encoder* instead processes the query and document as one input, allowing attention to connect words and phrases across the pair before producing a relevance score. That joint comparison can capture relationships lost when two independent vectors are reduced to one similarity value. It also prevents the document-side computation from being reused across queries: every candidate requires another model inference.

That cost places the cross-encoder near the end of the pipeline, inside one of the smallest candidate windows. The instrument below takes the fused RRF ranking from [Chapter 4](https://mutapa.dev/search-engineering/course/retrieval.md), selects only the top 5, 10, or 20 candidates, and rescores those query-document pairs. Each row traces one film from its RRF position to its cross-encoder position, while a gold star marks a labeled relevant result. Changing the window determines which candidates the model can inspect rather than how the model scores them.

Begin with “M3GAN” or “shark attacks a beach town”. In both cases, the reranker preserves the labeled film near the top. Then select “space movie where the dad ages slowly”. With a candidate window of 10, the model moves *Boyhood* above *Interstellar*, favoring the aging theme while losing the request's space context. For “robot falls in love”, it promotes *Vertigo* and demotes *Her*. The model used here was trained for general web passage ranking, so these movie-query failures are evidence of *training mismatch* rather than evidence that joint scoring is inherently better or worse.

> Interactive instrument: Reranking lab. The interactive edition changes the depth and cost of a reranking stage so you can inspect which candidates receive expensive attention.

The score itself does not reveal whether the model's training data matches the current domain. That must be measured on representative queries and judgments, just as the rerank window must be tested for candidate coverage. A team can respond to a mismatch by selecting a domain-appropriate reranker, fine-tuning with its own labeled pairs, or leaving the cross-encoder out when it consistently degrades the baseline.

Even a well-matched cross-encoder judges only the text it receives. Availability, recency, popularity, price, and distance remain invisible unless they are encoded into that input, and forcing structured features into prose makes their influence harder to control. The cross-encoder score is therefore one semantic feature in a broader ranking decision. The next section develops a model that can combine it with the behavioral and domain features introduced above.

## Learn ranking weights from judgments

[Learning to rank](https://en.wikipedia.org/wiki/Learning_to_rank) turns the final ordering into a [supervised learning](https://en.wikipedia.org/wiki/Supervised_learning) problem. Its training data groups documents by query and assigns each one a relevance grade. For every query-document pair, the system records features such as field-level BM25 scores, embedding similarity, the cross-encoder score, popularity, recency, availability, price, or distance. The model learns how those signals should interact instead of relying on the equal hand-set weights introduced in [Chapter 1](https://mutapa.dev/search-engineering/course/what-is-relevance.md).

The judgments deserve attention before the model does. Behavioral boosts can first be applied directly to frequent queries and evaluated in a controlled experiment. If those boosts improve outcomes, the team has evidence that the underlying events capture something useful about relevance. A learned ranker can then use that evidence with other features to generalize beyond the query-document pairs that accumulated enough interactions. Training immediately on untested labels hides mistakes inside the model and makes it harder to tell whether a failure came from the judgments, features, or learning objective.

The learning objective determines what the model treats as an error. A [pointwise](https://en.wikipedia.org/wiki/Learning_to_rank#Pointwise_approach) objective predicts each document's grade independently, even though the product ultimately needs an ordering. A [pairwise](https://en.wikipedia.org/wiki/Learning_to_rank#Pairwise_approach) objective instead learns preferences between two documents judged for the same query: document A should score above document B. At inference time, the model still assigns each document one score, but those scores were learned from within-query comparisons.

A [listwise](https://en.wikipedia.org/wiki/Learning_to_rank#Listwise_approach) objective accounts for the result list as a whole. [LambdaMART](https://www.microsoft.com/en-us/research/publication/from-ranknet-to-lambdarank-to-lambdamart-an-overview/) is a mature approach built from [gradient-boosted](https://en.wikipedia.org/wiki/Gradient_boosting) [decision trees](https://en.wikipedia.org/wiki/Decision_tree_learning). It gives more weight to pairwise corrections that would produce a larger improvement in a list metric such as [nDCG](https://en.wikipedia.org/wiki/Discounted_cumulative_gain). Moving a relevant document from rank 20 to 19 matters less than moving it from rank 2 to 1, so the training signal reflects the positions users are most likely to experience.

Tree-based rankers remain useful because they combine heterogeneous scalar features and learn conditional relationships among them. Distance may matter strongly for a local service and carry no value for a remote job; recency may dominate news and contribute little to an archival search. Where the latency budget permits cross-encoder inference, its semantic score can become one input to a final tree-based or linear composition over that narrow window. A cheaper learned ranker used earlier in the pipeline would rely on features available at its wider depth.

Two data boundaries keep that model honest. Train and test sets must be split by query so examples derived from one request do not appear on both sides of the evaluation. Dynamic features must also be recorded when the impression or interaction occurs. Recomputing a product's price, availability, or popularity months later attaches the judgment to a different state of the world. With those boundaries preserved, the learned ranker can serve as an intermediate phase: richer than a fixed boost, but inexpensive enough to score a wider window than the cross-encoder.

## Allocate the latency budget

Candidate windows allocate model computation, but dense retrieval introduces another cost: the memory and bandwidth required to search stored vectors. [Chapter 2](https://mutapa.dev/search-engineering/course/corpus-and-indexing.md) introduced quantization as an indexing decision. A 384-dimensional vector occupies 1,536 bytes in float32, 384 bytes in int8, or 48 bytes when reduced to one bit per dimension. Lower precision allows more vectors to remain in memory and makes the first comparison cheaper, but the compressed representation can change which documents appear nearest.

The instrument measures that change against the full-precision ranking. It selects 60 film vectors as queries, finds each one's top 10 neighbors with float32 cosine similarity, then checks how many of those neighbors binary search recovers. The resulting *recall@10* is an overlap measure between two retrieval methods. It does not report whether either list is relevant to a person. Direct binary search recovers an average of 44.8 percent of the float32 top ten. Select “Binary”, then choose “Over-fetch + rescore”, and watch the overlap measure change.

> Interactive instrument: Quantization dial. The interactive edition changes vector precision and over-fetch depth, then shows the resulting memory, latency, and recall tradeoff.

With “Over-fetch + rescore”, the first pass retrieves 40 candidates using the one-bit vectors instead of returning its top 10 directly. A second pass rescores those 40 candidates with the retained float32 vectors and returns the best 10. On this sample, overlap with the float32 baseline rises from 0.448 to 0.832. The second pass can repair the order only when the missing neighbor survived inside the wider binary window.

That recovery is a budget decision. A wider first pass improves the chance that full-precision rescoring can restore the baseline neighbors, but it increases candidate transfer, memory access, and reranking latency. The appropriate depth comes from the recovery curve measured on representative queries under the system's latency target. The full-precision vectors must also remain available for rescoring; discarding them at index time makes the compression loss permanent. Quantization therefore fits the chapter's larger pattern: use a cheap representation broadly, preserve a more faithful representation, and spend its cost only on the candidates the first phase can afford to pass forward.

## Correct bias in behavioral labels

The learning-to-rank section assumed that every query-document pair had a trustworthy relevance grade. *Click logs* record the query, the displayed results, their positions, and the user's subsequent actions, and they appear to provide those grades at scale. But a click occurs only after the system has chosen what to display and where to place it. Treating raw [click-through rate](https://en.wikipedia.org/wiki/Click-through_rate), clicks divided by displayed results, as relevance therefore carries the previous ranker's decisions into the next model.

The first distortion is *position bias*, the tendency for a result's position to affect the chance that a user notices and clicks it. Results near the top receive more attention, so they collect more clicks even when lower results would be equally useful. Dividing clicks by all *impressions*, meaning every rendered result, does not correct that imbalance because rendering does not prove examination. A [Simplified Dynamic Bayesian Network (SDBN)](https://dl.acm.org/doi/10.1145/1526709.1526711) is a click model, a probabilistic account of how examination, attraction, clicks, and satisfaction produce the observed event sequence. Its simplified examination rule uses a narrower denominator: within one search session, it treats results at or above the last click as examined and excludes results below that point.

Under this model, a click at rank 19 carries evidence because the user examined deeply enough to reach it. The grade is based on clicks among estimated examinations rather than among every rendered result. A *search session* here begins with one query and includes its result interactions until the query changes or the search ends. The assumption remains imperfect because a zero-click session reveals no last examined position. Scroll, hover, or dwell *telemetry*, interface events recorded for later analysis, can provide additional evidence, but each event still needs an explicit interpretation.

The corrected ratio also needs uncertainty. One click from one estimated examination and 100 clicks from 100 examinations both produce 1.0, despite supporting very different confidence. A [Beta distribution](https://en.wikipedia.org/wiki/Beta_distribution) supplies a *prior*, an initial assumption about the click probability before the current observations, with parameters $\alpha_0$ and $\beta_0$. After observing `c` clicks across `e` examinations, [Bayesian inference](https://en.wikipedia.org/wiki/Bayesian_inference) updates that assumption. The resulting *posterior mean*, the estimated click probability after incorporating the observations, becomes

$$
\frac{\alpha_0 + c}{\alpha_0 + \beta_0 + e}
$$

This mean can serve as the relevance grade, while the *posterior variance* measures how much uncertainty remains after the observations. A learning-to-rank model can convert that confidence into a *sample weight*, a multiplier controlling how strongly one training example contributes to the learning objective. A large, consistent history can therefore influence training more than an isolated click. The prior determines how strongly sparse observations move toward its default grade, so its parameters must be fitted to the traffic and outcome rates of the product.

These transformations make the judgment list a model of user behavior rather than a neutral record. Before training a ranker, a team can apply the derived grades directly as a temporary boost and evaluate them in an [A/B test](https://en.wikipedia.org/wiki/A/B_testing), a controlled experiment that compares the changed ranking with the existing one on separate traffic groups. If the labels produce a worse ordering on their own, a learned model will usually make the failure harder to diagnose rather than correct it.

Examination correction still covers only documents the current ranker chose to show. This second distortion, *presentation bias*, leaves no behavioral evidence for unseen candidates and can make a click-trained model reproduce the ranking that generated its labels. Controlled [exploration](https://en.wikipedia.org/wiki/Exploration-exploitation_dilemma) creates that missing evidence by exposing a small, guarded share of traffic to promising but uncertain candidates. The resulting observations can reveal useful regions of the *feature space*, combinations of feature values absent from the existing training examples, allowing later training rounds to learn from more than the current system's preferences.

## Compose retrieval and ranking stages

Fusion now has a second meaning. [Chapter 4](https://mutapa.dev/search-engineering/course/retrieval.md) fused lexical and dense rankings so the candidate set could preserve evidence found by either representation. Ranking must fuse different kinds of evidence about those candidates: retrieval scores, behavioral history, domain constraints, learned feature interactions, and joint query-document judgments. The first fusion improves what can survive retrieval; the second decides what deserves attention at the top of the list.

The complete path follows the cost of that evidence. Lexical and dense retrieval search broadly, using compressed representations where the memory budget requires them, then merge their candidate sets. A middle phase applies inexpensive behavioral and domain features, often through a tree-based ranker, to a smaller window. A cross-encoder jointly scores only the narrow set that survives. If recency, availability, or other structured evidence must still affect the final order, a lightweight final composition combines those values with the cross-encoder score over that same small set. Each boundary is sized from measured candidate coverage and latency rather than inherited defaults.

Every signal also needs one owner in the final score. A click-derived value can act as a direct boost for frequent queries or enter a learned ranker as a feature. Applying the same aggregation in both places counts the behavior twice, makes its effective weight difficult to interpret, and strengthens the feedback loop that produced the signal. The same discipline applies to retrieval and model scores: record where each value enters, how it is normalized, and which phase is allowed to change its influence.

For the opening space-dad request, retrieval succeeds when *Interstellar* reaches the candidate set; ranking succeeds when the available evidence places it ahead of weaker alternatives without exceeding the latency budget. The output is a short, ordered set of documents, together with the recorded scores and feature values needed to inspect that decision. A person may read that list directly, or a language model may receive it as evidence. [Chapter 6](https://mutapa.dev/search-engineering/course/generation.md) examines how generation should use that evidence, cite it, and decline to answer when the ranked candidates do not support a response.