Gen AI

Hybrid Search

Dense retrieval finds semantically related passages while lexical retrieval finds exact rare-term matches, and combining their two ranked lists catches cases either method alone would miss.

Dense retrieval finds semantically related passages while lexical retrieval finds exact rare-term matches, and reciprocal rank fusion combines both ranked lists into one, catching cases either method alone would miss

Why Does This Exist?

Dense retrieval, built on embeddings, excels at matching meaning even when the exact wording differs — "how do I reset my password" successfully finds a passage titled "forgot login credentials," because the two are close in vector space despite sharing almost no words. But that same strength creates a specific weakness: dense retrieval can genuinely struggle with rare, specific terms — a part number, an exact error code, an uncommon proper noun — precisely because a model trained on broad text may not have learned a sharp, distinctive embedding for something it saw rarely, if at all, during training.

Lexical retrieval — the older keyword-matching approach, typically implemented with algorithms like BM25 — has exactly the opposite profile. It's excellent at exact and near-exact term matches, including rare terms it's never "seen" in any learned sense, because it's matching literal tokens rather than learned meaning. But it fails on genuine paraphrase, the exact case dense retrieval handles well. Hybrid search exists because these two failure modes don't overlap, and combining both retrieval methods covers cases where either one alone would come up empty.

Think of It Like This

Asking two different experts the same question

Imagine consulting two different experts about where to find something: one is excellent at understanding the gist of a vague, loosely worded request and pointing you toward generally the right area, even when you can't quite articulate the exact term for what you're looking for. The other is excellent at exact-match lookups — give them a precise code or specific name and they'll find it immediately, but a vaguely worded, paraphrased request leaves them stuck, because they're matching on the literal words you used, not the underlying intent.

Neither expert alone reliably covers every kind of question you might ask. Consulting both and combining their two separate answers into one final recommendation catches cases where either expert individually would have failed — which is exactly the logic behind hybrid search.

How It Actually Works

Two independent rankings of the same query

The diagram above shows the structure directly: the same query runs through two entirely separate retrieval systems in parallel. Dense retrieval embeds the query and searches an embedding index, producing a ranked list ordered by semantic similarity. Lexical retrieval — most commonly using BM25, a refinement of simple term-frequency matching (the same family as bag of words) that accounts for term rarity and document length — searches the same corpus based on literal token overlap, producing its own independently-ranked list ordered by keyword relevance.

Combining two rankings that use incomparable scores

The genuine difficulty in hybrid search isn't running two retrieval methods — it's combining their two outputs into one final ranking, when the two methods' raw scores aren't directly comparable to each other; a cosine similarity of 0.8 and a BM25 score of 12.4 don't sit on any shared scale that would let you simply average them meaningfully. Reciprocal rank fusion sidesteps this by ignoring the raw scores entirely and working only with each method's rank — a document's position in each list, not its score:

\text{RRF}(d) = \sum_{\text{list} \in \{\text{dense}, \text{lexical}\}} \frac{1}{k + \text{rank}_{\text{list}}(d)}

$k$ is a small constant (commonly around 60) that softens the impact of very high individual ranks. A document ranked highly in either list contributes a large term to its combined score; a document that ranks well in both lists accumulates two large contributions and rises to the top of the fused ranking. Because this formula only ever looks at rank position, never the underlying score's scale or units, it sidesteps the score-comparability problem entirely.

What hybrid search recovers that neither method alone would

A query containing both a specific rare term and a genuinely paraphrased concept — "error code E4021 keeps stopping my checkout" — plays to both retrieval methods' individual strengths simultaneously: lexical retrieval can lock onto the exact term "E4021" that dense retrieval might embed only loosely, while dense retrieval can surface passages about "checkout failures" that never mention the term "stopping" at all. Fusing the two ranked lists recovers documents that would rank well in one list but poorly in the other, results that a single-method search would likely miss or under-rank.

Show Me the Code

Reciprocal rank fusion combining two independently-ranked lists, computed directly from rank position rather than any raw score.

1def reciprocal_rank_fusion(ranked_lists: list[list[str]], k: int = 60) -> list[str]:2    scores: dict[str, float] = {}3    for ranked_list in ranked_lists:4        for rank, doc_id in enumerate(ranked_list, start=1):     # rank starts at 1, not 05            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)6    return sorted(scores, key=lambda d: scores[d], reverse=True)7
8
9dense_ranking = ["doc_a", "doc_c", "doc_b", "doc_d"]      # ranked by semantic similarity10lexical_ranking = ["doc_b", "doc_a", "doc_e", "doc_c"]     # ranked by exact term match11
12fused = reciprocal_rank_fusion([dense_ranking, lexical_ranking])13print(fused)  # -> ['doc_a', 'doc_b', 'doc_c', 'doc_d', 'doc_e'] -- doc_a and doc_b rank well in both lists

doc_a and doc_b each rank highly in one list and reasonably in the other, so they rise to the top of the fused ranking — doc_e, appearing only in the lexical list and ranked low there, ends up at the bottom, exactly reflecting that it's a weaker candidate by either method.

Watch Out For

Averaging raw scores instead of fusing by rank

Combining a dense retrieval score and a lexical retrieval score by simply averaging or summing them directly is a common but flawed shortcut, because the two scores don't share a scale — a BM25 score's magnitude depends on corpus statistics and query length in ways cosine similarity's bounded $[-1, 1]$ range never does. Whichever method happens to produce numerically larger raw scores ends up dominating the combined ranking, for reasons that have nothing to do with actual relevance. Fusing by rank position, as reciprocal rank fusion does, avoids this scale mismatch entirely.

Assuming hybrid search is strictly better and skipping evaluation

Running two retrieval methods and fusing their results generally helps, but it isn't a guaranteed universal improvement over the better of the two methods alone for every specific query distribution — added complexity and latency come with running two systems instead of one, and the actual quality gain depends on how much the two methods' failure modes genuinely complement each other for the specific corpus and query patterns in question. Measuring hybrid search against each individual method, on real queries, is worth doing rather than assuming the combination always wins.

The Quick Version

Dense retrieval matches meaning even through paraphrase but can struggle with rare, specific terms; lexical retrieval does the reverse.
Hybrid search runs both methods independently and combines their two ranked lists into one.
Reciprocal rank fusion combines lists using each document's rank position, not its raw score, avoiding the problem of incomparable scales.
A document ranking well in either list contributes strongly to the fused ranking; ranking well in both compounds that contribution.
Hybrid search generally helps but isn't automatically superior for every query pattern — it's worth measuring against each individual method.