A RAG system shows that with 15 retrieved chunks Claude's answer quality peaks, but with 25 chunks quality degrades — Claude gives less specific, more generic answers. What phenomenon explains this, and what is the architectural mitigation?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — the 'lost-in-the-middle' phenomenon is well-documented in LLM research: models attend more to content at the beginning and end of their context window and less to content in the middle. With 25 chunks, the most relevant chunks may appear mid-context, reducing their influence compared to 15 chunks where relevant content is more prominently positioned.
Full explanation below image
Full Explanation
The 'lost-in-the-middle' phenomenon is well-documented in LLM research: models attend more to content at the beginning and end of their context window and less to content in the middle. With 25 chunks, the most relevant chunks may appear mid-context, reducing their influence compared to 15 chunks where relevant content is more prominently positioned. The mitigation is two-fold: (1) use a reranker to rank chunks by query relevance, then (2) strategically order context with highest-relevance chunks at the beginning and end of the context block. Option A is incorrect — 25 chunks of typical 300-500 token size totals 7,500-12,500 tokens, well within Claude's 200K window. Option C is incorrect — adding more chunks to the retrieval context doesn't affect embedding computation quality (embeddings are computed separately at indexing time). Option D confuses input and output tokens — chunk context affects input token count, not output token budget.