A legal research assistant uses RAG to retrieve relevant case law. Attorneys report that keyword-specific queries (searching for a specific statute number, '42 U.S.C. § 1983') return irrelevant cases that are semantically similar but do not cite the exact statute. The system uses only dense vector retrieval. What change best fixes this?
Select an answer to reveal the explanation.
Short Explanation
You need hybrid retrieval here. BM25 catches exact statute numbers and citations, while vector search handles semantic similarity, and Reciprocal Rank Fusion puts exact matches back on top. That fixes keyword-specific queries without throwing away semantic search.
Full Explanation
The correct change is to add BM25 keyword retrieval in parallel with vector retrieval and fuse the results using Reciprocal Rank Fusion (RRF), which combines the strengths of exact keyword matching and semantic similarity. This addresses the failure mode because BM25 excels at exact term matching, such as statute numbers, case citations, and named parties, while vector retrieval excels at semantic similarity. RRF produces a ranked list that rewards documents appearing highly in both lists, ensuring exact-match documents surface even when they are not the closest semantic neighbors. Increasing the number of retrieved documents from top-5 to top-20 is wrong because it increases recall but does not solve the precision problem; the statute-specific documents may still rank below semantically similar but irrelevant cases. Fine-tuning the embedding model on legal documents is wrong because it would require legal training data and compute investment, and may not adequately capture exact-match behavior for specific numeric identifiers like statute numbers; BM25 handles this natively. Lowering the similarity threshold for vector retrieval is wrong because it accepts more documents of lower relevance without specifically prioritizing exact-match documents.