A knowledge management system has a corpus of 5,000 internal documents totaling approximately 50 million tokens. Users query the corpus for specific information. An architect is deciding between two approaches: (A) stuff all documents into Claude's context window using a very large context model, or (B) implement RAG to retrieve the 5-10 most relevant documents before calling Claude. Which factors most strongly favor approach B?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — d is correct because three factors converge to favor RAG: (1) 50M tokens vastly exceeds even the largest commercially available context windows (200K-1M tokens); (2) per-query cost with context stuffing would be catastrophically expensive — retrieving 5-10 documents reduces input tokens by 99.9%; (3) attention dilution is a documented phenomenon where relevant passages buried in massive context receive less attention weight than when they appear in focused context. A is wrong because even if a context window large enough existed, stuffing irrelevant documents degrades response quality and is economically prohibitive.
Full explanation below image
Full Explanation
D is correct because three factors converge to favor RAG: (1) 50M tokens vastly exceeds even the largest commercially available context windows (200K-1M tokens); (2) per-query cost with context stuffing would be catastrophically expensive — retrieving 5-10 documents reduces input tokens by 99.9%; (3) attention dilution is a documented phenomenon where relevant passages buried in massive context receive less attention weight than when they appear in focused context. A is wrong because even if a context window large enough existed, stuffing irrelevant documents degrades response quality and is economically prohibitive. C is wrong because document structure (structured vs. unstructured) does not determine the appropriate retrieval architecture — token count and cost are the primary factors. B is wrong because while incremental index updates are a benefit of RAG for frequently updated corpora, the primary reasons to use RAG (token limits, cost, attention dilution) apply regardless of update frequency.