The charter-desk assistant retrieves the top 25 clause chunks for every question because the team assumed more evidence is always better. Answer quality has fallen: the model now frequently cites a clause that is only loosely related to the question. What is the best explanation and remedy?
Select an answer to reveal the explanation.
Short Explanation
You’re better served by a few strong passages than a crowd of weak matches: too many loosely related chunks can dilute the context and tempt the model to cite weak evidence. Improve precision by retrieving fewer results and reranking the ones that remain.
Full Explanation
Retrieving 25 chunks can increase recall, but it also admits low-relevance passages that dilute the context and invite the model to rely on weak evidence. Reduce the number of results and rerank the remaining candidates so the strongest evidence is prominent. Increasing chunk size while keeping 25 results does not restore precision; larger chunks consume more context and may bury the relevant passage. Replacing the embedding model with a larger one may improve semantic matching, but retaining all 25 results still exposes the model to weakly related evidence. Instructing the model to ignore irrelevant chunks may help at the margin, but it treats a retrieval problem as a generation problem and still incurs the token cost and latency of supplying 25 passages. For the exam, recognize overly long context as a failure mode: more retrieved material is not automatically better evidence. As a practical check, inspect the retrieved results for representative questions and see whether the lower-ranked chunks genuinely support an answer; use that evidence to set a smaller result count and assess whether reranking puts the most relevant passages first.