Skip to content
Blog

Hybrid Search: Combining BM25 Keyword Scores With Vector Similarity

How hybrid search blends BM25 keyword ranking with dense vector similarity to catch exact terms and paraphrase alike, plus fusion techniques.

Published on • November 22, 2026

AI Assistant

Introduction

Keyword search finds ERROR_429 flawlessly and misses “rate limit” paraphrases. Vector search does the reverse. Hybrid search runs both and fuses the rankings — which is why it has become the default retrieval mode for serious search products and RAG pipelines.

Why Neither Alone Is Enough

Query typeBM25Vector
”how to reset my password” vs doc “credential recovery procedure”❌ no overlap✅ matches
”ECONNRESET” error code✅ exact hit❌ diluted in semantic space
Product SKU “AX-9912-B”✅ exact❌ often missing
”movies like Inception but funnier”❌✅ captures intent

Dense vectors generalise but blur rare tokens; BM25 is precise but brittle to paraphrase. Their failure modes are almost complementary — the definition of a good ensemble.

The Hybrid Architecture

                ┌─ BM25 (inverted index) ──── top-k keyword ranks ─┐
query ──────────┤                                                  ├─ fusion → top-k
                └─ embed → ANN index ──────── top-k vector ranks ──┘
  1. The query hits both retrieval legs in parallel.
  2. Each returns its own ranked top-k (often overfetch to k×5–k×10).
  3. A fusion method merges the two rankings into one.

Most systems (Elasticsearch, OpenSearch, Weaviate, Qdrant, Vespa, Pinecone) now ship this natively; LangChain-style RAG stacks compose it from retrievers.

Fusion Techniques

Score normalisation + weighted sum

Rescale each leg’s scores (min-max or z-score) to a common range, then score = α·bm25 + (1−α)·vector.

  • ✅ Intuitive, tunable with one α
  • ❌ score distributions differ per query; normalisation is fiddly; BM25 is unbounded while cosine is [-1,1]

Reciprocal rank fusion (RRF)

Ignore raw scores; use ranks only:

RRF(d) = Σ 1 / (rank_leg(d) + c)     # c ≈ 60
  • ✅ No score calibration needed — robust default
  • ✅ Works even when the two legs’ score scales are incomparable
  • (RRF gets its own dedicated treatment in this series — see topic #21)

Learn-to-rank / rerank stage

Fuse candidates, then pass the union to a cross-encoder reranker that scores query–document pairs directly. This is the highest-quality pattern: hybrid recall for candidates, reranker precision for final ordering.

Practical Tuning

  1. Overfetch both legs (k×10) so fusion has material to work with — fusing only top-5 too early starves the blend.
  2. Set α by query type: exact-ID-heavy corpora lean BM25 (α up); paraphrase-heavy lean vector. Some systems switch legs per query via a classifier.
  3. Watch tokenisation for rare identifiers — ensure BM25 indexes SKUs, error codes, and part numbers as tokens (or add a keyword subfield).
  4. Deduplicate — the same chunk often appears in both legs; fuse by document ID.
  5. Evaluate end-to-end: hybrid should beat both single legs on your eval set, or it’s complexity for nothing.

Hybrid in RAG

In retrieval-augmented generation, hybrid search reduces the two classic failure modes: missing context (model hallucinates because retrieval failed) and wrong context (irrelevant chunks poisoning the prompt). Combined with metadata filters and a reranker, it is the current best-practice retrieval stack.

Conclusion

Hybrid search is not “two searches, pick one” — it is a fusion problem: run BM25 for exactness, vectors for meaning, then blend ranks (RRF is the safe default) and optionally rerank. When the two legs’ weaknesses are each other’s strengths, the ensemble wins.

References