The Core Idea: There are two basic ways to find relevant text. Sparse retrieval matches the exact words of the query, and dense retrieval matches its meaning through embeddings. Each fails where the other succeeds: keywords miss paraphrases, and embeddings miss exact codes and rare names. Hybrid search runs both and fuses their rankings, which is why it is the default in many production systems.
1. Beginner: Two Ways to Find Text
Picture two assistants. The first has the index at the back of a book: give it a word and it lists every page that contains that exact word. The second is a well-read colleague who understands what you mean even if you phrase it differently, but who may forget the exact spelling of a part number.
Sparse retrieval
Matches words. Fast, exact, easy to explain. Called sparse because each text is a huge vector that is mostly zeros, one slot per possible word.
Dense retrieval
Matches meaning. Uses embeddings, short vectors with every slot filled with a number. Called dense for that reason.
Key point: neither approach is better in general. The real question is what your users ask: concepts and paraphrases, or exact names, codes and phrases.
2. Beginner: Sparse Retrieval and BM25
Keyword search is built on an inverted index: for every word, a list of the documents that contain it. At query time the system looks up the query words and scores the documents that contain them. The standard scoring function is BM25.
BM25 rewards a document when it contains the query word often (term frequency), when that word is rare across the whole collection (inverse document frequency), and it avoids favoring long documents just for being long:
Here counts the word in document , is the document length, avgdl is the average length, and and are tuning constants, commonly around 1.2 to 2 and 0.75. A rare word such as an error code gets a large IDF, so a single exact match can dominate.
Strengths and weakness: excellent for exact terms, names, identifiers and quoted phrases, and it needs no model. Its weakness is vocabulary mismatch: a query about a lost login finds nothing if the document says credentials recovery.
3. Intermediate: Dense Retrieval
Dense retrieval embeds the query and the chunks and returns the nearest vectors. It bridges vocabulary gaps, because texts with the same meaning land close together.
| Aspect | Sparse (BM25) | Dense (embeddings) |
|---|---|---|
| Matches | Exact words | Meaning |
| Paraphrases and synonyms | Weak | Strong |
| Codes, names, rare terms | Strong | Weak |
| Needs a trained model | No | Yes |
| Explainable | Yes, shows matched words | Harder |
| Index and compute cost | Light | Heavier |
4. Intermediate: Hybrid Search and Reciprocal Rank Fusion
Hybrid search sends the query to both retrievers and merges the results. The difficulty is that their scores live on different scales: a BM25 score of 12 and a cosine similarity of 0.8 cannot be added meaningfully. A robust fix is to combine ranks instead of scores. Reciprocal Rank Fusion (RRF) gives every document a point value that shrinks with its rank in each list, then adds the points:
Here is the set of ranked lists and is a smoothing constant, often 60. A document that appears near the top of both lists beats one that is first in only one. A document found by only one retriever still gets credit, so each method can rescue the other.
Alternative: weighted score fusion first normalizes every score to a common range and then takes a weighted average. It can be tuned more finely, but it needs careful normalization and weights. RRF is popular because it works well with no tuning.
5. Advanced: Choosing and Tuning the Mix
Let query types guide the design. Look at real logs and group questions before you decide.
| Query type | Example | Better retriever |
|---|---|---|
| Conceptual | How do I get my money back? | Dense |
| Exact identifier | Error E4021 | Sparse |
| Mixed | Why does E4021 appear during checkout? | Hybrid |
| Quoted phrase or name | The Atlas project budget | Sparse or hybrid |
Baseline. Start with BM25. It is cheap and a stronger baseline than many expect.
Add dense. Measure whether embeddings find relevant chunks that BM25 misses.
Fuse. Use RRF first, and tune weights only if evaluation shows a gain.
Check the details. Tokenization, lowercasing and stemming settings in the keyword index change results more than people expect.
Some newer approaches learn which words matter, producing sparse vectors with learned weights. They keep the exactness of keywords while adding some semantic reach, and they are worth knowing about once the basics work.
6. Visual Explanation: Where Each Retriever Wins
Panel A compares the three approaches. Panel B shows real results from the code below for three queries. Panel C is a query-type explorer.
Panel A · Three ApproachesStrengths side by side
Sparse
Words
Inverted index
↓
BM25 scoring
↓
Great for exact terms
Dense
Meaning
Embeddings
↓
Nearest vectors
↓
Great for paraphrases
Hybrid
Both
Run both retrievers
↓
Fuse with RRF
↓
Covers both failure types
Panel B · Results from the CodeTop results per method
| Query | Sparse | Dense | Hybrid |
|---|---|---|---|
| I lost my access | nothing found | d2, d1 | d2, d1 |
| E4021 | d3 | nothing found | d3 |
| forgot password | d2, d1 | d2, d1 | d2, d1 |
Each single method fails on one query, and hybrid never fails here
Panel C · Query ExplorerClick a query type to open it
Click to see why RRF uses ranks and not scores▼
Scores from different retrievers are on incompatible scales, so adding them is meaningless. Ranks are comparable everywhere, which makes fusion robust without any tuning.
How to read it: Panel A names the options, Panel B shows that each single method has a blind spot, and Panel C turns that into a rule of thumb for your own queries.
7. Python Implementation: BM25, a Toy Dense Retriever and RRF
The program implements BM25 from scratch for the sparse side. For the dense side it uses a small table that maps words to shared concepts, so texts with different words but the same meaning get similar vectors. This stands in for a real embedding model. Reciprocal rank fusion then merges the two rankings. Three queries show the three situations from the lesson.
hybrid_retrieval.pyPython 3.10+ · Standard library only
import mathfrom collections import Counter documents = [ ("d1", "how to reset your password using the account page"), ("d2", "forgot login credentials recovery steps"), ("d3", "error code E4021 means the payment failed"), ("d4", "refund policy for returns within 30 days"), ("d5", "billing and invoice questions"),] def tokens(text): cleaned = "".join(ch if ch.isalnum() else " " for ch in text.lower()) return cleaned.split() # ---- Sparse: BM25 on exact words ----doc_tokens = [(doc_id, tokens(text)) for doc_id, text in documents]avg_len = sum(len(t) for d, t in doc_tokens) / len(doc_tokens) def idf(word): n = sum(1 for d, t in doc_tokens if word in t) return math.log(1 + (len(doc_tokens) - n + 0.5) / (n + 0.5)) def bm25(query, k1=1.5, b=0.75): scores = [] for doc_id, t in doc_tokens: score = 0.0 for word in set(tokens(query)): tf = t.count(word) if tf: score += idf(word) * tf * (k1 + 1) / (tf + k1 * (1 - b + b * len(t) / avg_len)) scores.append((score, doc_id)) return scores # ---- Dense (toy): words mapped to shared concepts ----concepts = dict([ ("password", "login"), ("credentials", "login"), ("login", "login"), ("account", "login"), ("forgot", "recover"), ("reset", "recover"), ("recovery", "recover"), ("forgotten", "recover"), ("lost", "recover"), ("access", "login"), ("payment", "money"), ("billing", "money"), ("invoice", "money"), ("refund", "money"),]) def concept_vector(text): return Counter(concepts[w] for w in tokens(text) if w in concepts) def cosine(a, b): dot = sum(a[k] * b[k] for k in a) na = math.sqrt(sum(v * v for v in a.values())) nb = math.sqrt(sum(v * v for v in b.values())) return dot / (na * nb) if na and nb else 0.0 def dense(query): q = concept_vector(query) return [(cosine(q, concept_vector(text)), doc_id) for doc_id, text in documents] # ---- Hybrid: reciprocal rank fusion ----def ranking(scores): ordered = sorted(scores, key=lambda s: (-s[0], s[1])) return [doc_id for score, doc_id in ordered if score > 0] def rrf(rankings, k=60): fused = Counter() for ranked in rankings: for position, doc_id in enumerate(ranked): fused[doc_id] += 1 / (k + position + 1) return [doc_id for doc_id, s in sorted(fused.items(), key=lambda x: (-x[1], x[0]))] for query in ["I lost my access", "E4021", "forgot password"]: sparse_rank = ranking(bm25(query)) dense_rank = ranking(dense(query)) print("Query:", query) print(" Sparse (BM25):", sparse_rank[:3]) print(" Dense (toy) :", dense_rank[:3]) print(" Hybrid (RRF) :", rrf([sparse_rank, dense_rank])[:3])Expected output
Query: I lost my access Sparse (BM25): [] Dense (toy) : ['d2', 'd1'] Hybrid (RRF) : ['d2', 'd1']Query: E4021 Sparse (BM25): ['d3'] Dense (toy) : [] Hybrid (RRF) : ['d3']Query: forgot password Sparse (BM25): ['d2', 'd1'] Dense (toy) : ['d2', 'd1'] Hybrid (RRF) : ['d2', 'd1']The query I lost my access shares no word with any document, so BM25 returns nothing, yet the concept table finds the account recovery document. The query E4021 is an exact code that the concept table has never seen, so only BM25 finds it. Hybrid search returns a sensible answer in both cases. When both methods agree, as in the third query, fusion simply confirms the ranking.
Knowledge Check
Q1. Which kind of query is hardest for dense retrieval?
Q2. Why is Reciprocal Rank Fusion based on ranks?
Q3. What is vocabulary mismatch?
Key Points
✓
Sparse retrieval matches exact words, usually with BM25 over an inverted index.
✓
Dense retrieval matches meaning with embeddings and handles paraphrases and synonyms.
✓
Sparse is strong on codes, names and rare terms; dense is strong on concepts; each fails where the other succeeds.
✓
Hybrid search runs both retrievers and fuses the results.
✓
Reciprocal rank fusion adds 1 divided by k plus rank for each list, with k often set to 60.
✓
Group real queries by type to decide the mix, and keep BM25 as a baseline to beat.
✓
Tokenization and stemming settings in keyword search noticeably change results.
The Big Picture
Dense and sparse retrieval are complements, not rivals. Combining them is cheap compared with the cost of a missed passage, so hybrid search is a strong default whenever your users mix concepts with exact terms.
Query → Sparse search + Dense search → Fuse ranks → Candidates
Up Next: Metadata Filtering and Advanced Retrieval. You will learn how to restrict retrieval by date, category, status and user permissions, and the difference between pre-filtering and post-filtering.