# Hybrid Retrieval with RRF

Search Engineering · Implementation Guide · Chapter 5

Interactive edition: https://mutapa.dev/search-engineering/build/hybrid-retrieval-with-rrf

Fuse keyword and vector rankings while preserving evidence from both retrievers.

## Outcome

This chapter adds two hybrid searches. `hybrid_search()` combines BM25 and vector ranks with Reciprocal Rank Fusion. `weighted_hybrid()` combines normalized scores with a lexical weight selected on training judgments.

Both functions preserve each source rank and score. A separate evaluation set then shows whether the selected weight transfers beyond the queries that chose it.

## Fusion needs its own rule

BM25 and vector retrieval expose different evidence. Chapter 4 showed a paraphrase that favored vectors, an agreement, and a semantic near-miss that favored lexical evidence.

Adding their raw scores would give the larger numeric scale more influence. BM25 scores are corpus-dependent and unbounded, while this guide's vector scores are cosine similarities between unit vectors.

RRF avoids that comparison by using rank positions. Weighted fusion first normalizes each list, then learns how much influence to give each source from explicit judgments.

Neither method guarantees an improvement on every query. Fusion can also promote a marginal document that both retrievers rank moderately above a relevant document found by only one.

The [retrieval runbook](https://mutapa.dev/search-engineering/reference/retrieval.md#score_distribution_mismatch) covers score-scale failures in deployed systems.

## Prerequisites

- Chapters 1 through 4 completed.
- BM25 `search()` from Chapter 2 working.
- Vector `vector_search()` from Chapter 4 working with the selected Chapter 3 artifact.
- The locked Python 3.14 environment active.

## Setup

This chapter adds no dependencies:

```bash
uv sync --python 3.14 --locked
source .venv/bin/activate
```

The examples run BM25 and vector retrieval sequentially. Parallel execution is an operational optimization that requires timeout, cancellation, and client-concurrency decisions outside this fusion mechanism.

## Step 1: Preserve candidate provenance

The collector keeps one contribution per document from each retriever. It records the source rank and raw score so a fused result remains diagnosable.

```python
def _collect(rankings: dict[str, list[dict]]) -> dict[str, Result]:
    candidates: dict[str, Result] = {}
    for source, ranking in rankings.items():
        seen: set[str] = set()
        for rank, document in enumerate(ranking, start=1):
            doc_id = str(document["doc_id"])
            if doc_id in seen:
                continue
            seen.add(doc_id)
            candidate = candidates.setdefault(
                doc_id,
                {
                    "doc_id": doc_id,
                    "title": str(document["title"]),
                    "body": str(document["body"]),
                },
            )
            candidate[f"{source}_rank"] = rank
            score_field = f"{source}_score"
            if score_field in document:
                candidate[score_field] = float(document[score_field])
    return candidates
```

The per-source `seen` set prevents one malformed ranking from contributing more than once for the same document.

## Step 2: Fuse ranks with RRF

For each candidate, RRF adds one reciprocal contribution from every list containing it:

```text
rrf_score(document) = sum(1 / (rank_constant + source_rank))
```

The original RRF experiments used `60` for the constant. This value reduces the gap between adjacent ranks; it is a starting configuration rather than a relevance guarantee.

```python
# code/chapters/chapter_04.py
def rrf(
    rankings: dict[str, list[dict]],
    rank_constant: int = 60,
) -> list[Result]:
    if rank_constant < 1:
        raise ValueError("rank_constant must be positive")

    candidates = _collect(rankings)
    for candidate in candidates.values():
        candidate["rrf_score"] = sum(
            1.0 / (rank_constant + int(candidate[f"{source}_rank"]))
            for source in rankings
            if f"{source}_rank" in candidate
        )
```

The final ordering uses fused score, best source rank, then document ID. The last key makes equal-score ordering independent of which retriever was processed first.

```python
return sorted(
    candidates.values(),
    key=lambda candidate: (
        -float(candidate["rrf_score"]),
        min(
            int(candidate[f"{source}_rank"])
            for source in rankings
            if f"{source}_rank" in candidate
        ),
        str(candidate["doc_id"]),
    ),
)
```

## Step 3: Retrieve a deeper candidate pool

`hybrid_search()` asks each retriever for more rows than it returns. This gives fusion a larger pool in which to find agreement.

```python
def hybrid_search(query: str, k: int = 10, overfetch: int = 3) -> list[Result]:
    if k < 1:
        return []
    if overfetch < 1:
        raise ValueError("overfetch must be positive")
    bm25_results, vector_results = retrieve(query, k * overfetch)
    return rrf({"bm25": bm25_results, "vector": vector_results})[:k]
```

This candidate-depth setting belongs to fusion. It is distinct from retrieving approximate or compressed candidates and rescoring them with full-precision vectors.

## Step 4: Inspect three fused rankings

The script reuses the Chapter 4 queries:

```python
compare_three("space movie where the dad ages slowly")
compare_three("locked jury room")
compare_three("dreams inside dreams")
```

For the paraphrase query, local BGE ranks *Interstellar* first but RRF ranks it fourth. Several less relevant films receive contributions from both lists and move above it.

For `locked jury room`, both retrievers rank *12 Angry Men* first, so RRF preserves it. For `dreams inside dreams`, agreement places *Inception* above the vector-only first result, *Inside Out*.

The first case is a useful warning. Consensus is a fusion signal rather than proof of relevance. Chapter 7 will evaluate the result set against judgments rather than isolated impressions.

## Step 5: Normalize scores for weighted fusion

Weighted fusion retains score differences that RRF discards. The implementation min-max normalizes each candidate list independently to `[0, 1]`.

```python
def normalize_scores(results: list[dict], score_field: str) -> dict[str, float]:
    if not results:
        return {}
    scores = np.asarray([result[score_field] for result in results], dtype=np.float64)
    low, high = float(scores.min()), float(scores.max())
    if np.isclose(low, high):
        return {str(result["doc_id"]): 0.5 for result in results}
    return {
        str(result["doc_id"]): float((result[score_field] - low) / (high - low))
        for result in results
    }
```

Min-max normalization is sensitive to the retrieved range. It makes this experiment coherent, but its behavior must still be evaluated on the target query distribution.

Only one free parameter is required. The vector weight is `1 - lexical_weight`, so both weights remain non-negative and sum to one.

```python
candidate["hybrid_score"] = (
    lexical_weight * lexical.get(doc_id, 0.0)
    + (1.0 - lexical_weight) * vector.get(doc_id, 0.0)
)
```

A document absent from one candidate list receives zero from that source. The returned record still includes every available source rank and raw score.

## Step 6: Select and evaluate the weight

The script includes eight training judgments and four evaluation judgments using real corpus IDs. The set is deliberately small and demonstrates the procedure; it cannot support a production quality claim.

```python
TRAIN_JUDGMENTS: list[Judgment] = [
    ("space movie where the dad ages slowly", "interstellar-2014"),
    ("locked jury room", "12-angry-men-1957"),
    ("dreams inside dreams", "inception-2010"),
    # Five more runnable judgments in chapter_04.py.
]

EVAL_JUDGMENTS: list[Judgment] = [
    ("knight plays chess with Death", "the-seventh-seal-1957"),
    ("a painter secretly makes a bride's portrait", "portrait-of-a-lady-on-fire-2019"),
    ("lonely robot cleans an abandoned Earth", "wall-e-2008"),
    ("crime family led by Don Corleone", "the-godfather-1972"),
]
```

Mean reciprocal rank, or MRR, averages the reciprocal position of the first judged relevant document. A missing document contributes zero.

The local BGE run scores `1.000` on both small sets for weighted fusion, and RRF also scores `1.000` on the evaluation set. The queries are easy enough that these equal scores establish execution rather than superiority.

The selector tests lexical weights from `0.0` through `1.0` in increments of `0.1`. A fixed grid is reproducible and avoids sampling equivalent pairs of independently scaled weights.

```python
def select_weight(judgments: list[Judgment] = TRAIN_JUDGMENTS) -> tuple[float, float]:
    candidates = [step / 10 for step in range(11)]
    scored = [
        (
            mean_reciprocal_rank(
                judgments,
                lambda query, k, weight=weight: weighted_hybrid(
                    query, k, lexical_weight=weight
                ),
            ),
            weight,
        )
        for weight in candidates
    ]
    train_mrr, lexical_weight = max(
        scored,
        key=lambda item: (item[0], -abs(item[1] - 0.5)),
    )
    return lexical_weight, train_mrr
```

The selected weight is evaluated once on `EVAL_JUDGMENTS`. Compare that result with RRF on the same evaluation queries. Do not choose the weight again after seeing evaluation performance.

## Run it

Use the embedding artifact selected in Chapter 4:

```bash
# Default OpenAI artifact
python code/chapters/chapter_04.py

# Local BGE artifact
EMBEDDINGS_FILE=embeddings-local.npz python code/chapters/chapter_04.py
```

The script prints the three comparisons, selected lexical weight, training MRR, evaluation RRF MRR, and evaluation weighted MRR.

## Success criteria

- A duplicate document within one source list contributes once.
- Every fused record retains available BM25 and vector ranks and scores.
- RRF scores are non-increasing, with deterministic ordering for ties.
- `rank_constant < 1` and `overfetch < 1` raise `ValueError`.
- The weight search examines the fixed grid and reports training performance separately from evaluation performance.
- The evaluation result is treated as evidence from four judgments rather than a general claim that one fusion method wins.

## Troubleshooting

- Results match one retriever: inspect the source ranks. The other retriever may have returned no candidates or only low-ranked candidates.
- Small RRF scores: their magnitude comes from the rank constant. Interpret their ordering rather than a fixed threshold.
- An unexpected consensus result: inspect `bm25_rank` and `vector_rank` before changing the constant or candidate depth.
- Weighted results change sharply with candidate depth: min-max normalization is responding to a different score range. Hold depth fixed during comparison.
- High training MRR and lower evaluation MRR: collect more judgments rather than retuning on the evaluation set.

## Production considerations

> Preserve source ranks, raw scores, model identifiers, candidate depth, and fusion parameters in retrieval traces. Without them, a fused ranking cannot be attributed to lexical retrieval, vector retrieval, or fusion.
>
> Authorization and tenant boundaries must constrain retrieval before candidates enter fusion. Ordinary relevance filters may run before or after retrieval, but measure their effect on candidate recall.
>
> Introduce a reranker only after labeled evaluation identifies errors that candidate fusion cannot resolve. Choose its candidate depth from measured recall, latency, and cost rather than a fixed tutorial value.
>
> If a search engine provides fusion, verify its rank convention, constant, candidate depth, tie handling, and filtering semantics against this reference implementation before replacing it.

## What you built

RRF now combines ranks without comparing incompatible raw scores. Weighted fusion makes the score transformation and lexical influence explicit.

The preserved provenance shows why each document moved. The separate judgment sets show whether a tuned choice transfers beyond the queries that selected it.

Chapter 6 sends the fused candidates to generation. Chapter 7 evaluates retrieval and generation more systematically, and Chapter 8 identifies the next extensions.

## Further reading

- [Reciprocal Rank Fusion paper](https://doi.org/10.1145/1571941.1572114)
- [Diagnose hybrid retrieval in an existing system](https://mutapa.dev/search-engineering/reference/retrieval.md)
- [Next chapter: Chapter 6, Generation](https://mutapa.dev/search-engineering/build/generation.md)