A knowledge management system chunks documents into 512-token chunks with 50-token overlap. Users ask questions that require synthesizing information spread across 8-10 chunks. Retrieved chunks are limited to top-5 by default. Engineers increase top-k to 20 to capture more chunks but find answer quality degrades because the LLM context is now full of marginally relevant chunks. What retrieval architecture addresses this more effectively?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because the problem is retrieval precision: ANN vector search retrieves semantically plausible candidates but lacks the fine-grained relevance discrimination needed to distinguish essential chunks from marginally relevant ones. A cross-encoder reranker evaluates query-document relevance with full attention over both, producing much higher precision scores.
Full explanation below image
Full Explanation
B is correct because the problem is retrieval precision: ANN vector search retrieves semantically plausible candidates but lacks the fine-grained relevance discrimination needed to distinguish essential chunks from marginally relevant ones. A cross-encoder reranker evaluates query-document relevance with full attention over both, producing much higher precision scores. Running it over 50 first-stage candidates and passing only the top-10 reranked results to the LLM gives the LLM a high-precision, high-coverage set. A is wrong because reducing chunk size to 256 tokens means each chunk holds less information; the system would need even more chunks to capture the same content, not fewer, and the precision problem is unchanged. C is wrong because max_tokens controls output length, not input processing; the problem is that marginally relevant input degrades the LLM's synthesis, which is not addressed by allowing longer output. D is wrong because meta-chunk retrieval is a useful approach for document-level selection but does not solve the within-document chunk precision problem; it also requires pre-generating summaries for every document, adding ingestion overhead.