A SaaS company operates a RAG system with a corpus of 2 million technical support documents. The chunking strategy uses 800-token fixed-size chunks with no semantic boundaries. Customer success reports that Claude's answers to troubleshooting questions often contain correct information but in the wrong order — instructions from step 3 are presented before step 1. Retrieval precision is high (correct documents retrieved) but step ordering is broken. What is the root cause and correct chunking fix?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — post-retrieval re-ordering by source document position is the most direct fix for the step-ordering problem. The root cause is that the retrieval system returns chunks in order of semantic similarity, not document position — step 3 may be more semantically similar to the query than step 1 and therefore ranks higher in retrieval results.
Full explanation below image
Full Explanation
Post-retrieval re-ordering by source document position is the most direct fix for the step-ordering problem. The root cause is that the retrieval system returns chunks in order of semantic similarity, not document position — step 3 may be more semantically similar to the query than step 1 and therefore ranks higher in retrieval results. When Claude receives chunks in similarity order (step 3, step 1, step 2), it synthesizes a response with that ordering. The fix: after retrieval, sort all chunks from the same source document by their original position (chunk index, page number, or character offset stored as metadata), then present to Claude in document order. Option A (semantic chunking) addresses boundary splitting but doesn't fix ordering — even semantically chunked documents return steps in similarity order, not sequential order. Option B (prepending 'Step N of M') helps Claude understand ordering once it has the chunks but doesn't fix that the retrieval returns them out of order — Claude may still receive them in the wrong order in its context. Option D (larger chunks) is counterproductive — 3,000-token chunks may include too much irrelevant content alongside the relevant steps, degrading response quality.