A SaaS company operates a RAG system with a corpus of 2 million technical support documents. The chunking strategy uses 800-token fixed-size chunks with no semantic boundaries. Customer success reports that Claude's answers to troubleshooting questions often contain correct information but in the wrong order — instructions from step 3 are presented before step 1. Retrieval precision is high (correct documents retrieved) but step ordering is broken. Procedures typically span 2,000-4,000 tokens, and all chunks of the relevant procedure are retrieved. What is the root cause and the correct fix?
Select an answer to reveal the explanation.
Short Explanation
It's like getting a deck of recipe cards in the order they matched your search, not the order you cook. All the right chunks show up, so you just sort them back into their original document position before Claude sees them.
Full Explanation
The root cause is ordering, not retrieval quality or chunk boundaries. Retrieval ranks chunks by similarity to the query, so a chunk describing step 3 can outrank the chunk with step 1. Because each procedure spans several 800-token chunks and all of them are retrieved, Claude receives the right content in the wrong sequence and writes its answer in that sequence. Sorting retrieved chunks from the same source document by stored position metadata, such as chunk index or character offset, before building the prompt restores the original order. Anthropic's contextual retrieval write-up notes that chunk size and boundaries affect retrieval performance, but performance is already high here; the problem appears after retrieval. Switching to sentence-boundary chunking with the same 800-token cap still splits a multi-thousand-token procedure across several chunks, so their ranking order is unchanged. Prepending step labels before embedding changes vectors rather than the order chunks are delivered. Raising chunk size to 3,000 tokens still cannot hold procedures up to 4,000 tokens and dilutes relevance for shorter queries. On the exam, separate retrieval failures from presentation failures. Operationally, store document ID and chunk index on every chunk so reordering is a simple sort.