Day 17: Why hybrid search is often the practical default
Why combining retrieval signals is often more practical than betting on one search method.
Choosing between keyword and semantic retrieval means choosing which failure mode you would rather live with.
Hybrid search declines the choice. Run both, merge the results, cover each system's blind spot with the other's strength. It is the least fashionable answer in modern retrieval and, for most production systems, the correct one.
The part nobody warns you about
"Run both and combine" hides the actual engineering, which is the combine.
Two ranked lists do not merge themselves. Keyword scores and vector similarities live on completely different scales — a BM25 score of 14.2 and a cosine similarity of 0.83 have no natural relationship, so something has to reconcile them. Usually that is Reciprocal Rank Fusion, which sidesteps the scale problem by using positions instead of scores, or weighted normalised scores if you want more control.
Either way, you have just added a component that decides what the model sees, and its defaults were tuned on somebody else's corpus. The fusion step is a first-class part of your retrieval quality, and it needs to be tested like one.
Three things to get right before shipping it
- Deduplicate deliberately. The same chunk will surface from both paths. Decide whether that agreement should boost its rank (usually yes, two independent signals agreeing is real evidence) or simply collapse to one entry.
- Budget the latency. Two searches plus fusion plus, typically, a reranker on top. Each step is defensible; the total is what your user experiences. Measure the p95, not the average.
- Log which path won. When retrieval misbehaves, the first question is which leg failed. Without per-path attribution in your traces you are debugging a merged result with no way to see its inputs.
Is it worth the moving parts?
Usually, yes, and the sanity check is easy. Ask anyone running pure vector search what happens when a user pastes invoice #88231.
But make it earn the complexity on your data. Build a small evaluation set drawn from real queries, run keyword-only, semantic-only, and hybrid against it, and look at the difference. Sometimes the gain is dramatic. Sometimes your corpus turns out to be almost entirely natural language and semantic-only is genuinely fine, in which case you just saved yourself a component to maintain.
That is a measurement, not a preference. Treat it as one.
Hybrid search is boring in the way that load balancers are boring: not a talking point, just a system that stops failing in a category of ways you would otherwise discover one user at a time.
Day 17 of 60 Days of Production AI Systems.
If you run hybrid retrieval, when did anyone last look at what your fusion step is actually rewarding?
Previous: Day 16: Why keyword search and semantic search both matter
Next: Day 18: Why relevant data can still be unauthorized data
Day 17 of 60 · Retrieval Foundations (chapter 3 of 10)
