A transit authority's RAG-based schedule assistant sometimes returns answers built from irrelevant retrieved passages, and sometimes misses relevant schedule details that were split across two separate retrieved chunks. The team tunes chunk size and top-k retrieval count to address this. Why are these the right parameters to adjust here?
Select an answer to reveal the explanation.
Short Explanation
Picture cutting a reference book into index cards versus whole chapters — cut them too small and a fact gets split across two cards; cut them too large and irrelevant text rides along with the answer. Chunk size decides how documents get sliced for retrieval, and top-k decides how many of those slices get pulled back for a given question. Tuning both together is exactly how you fix both missed context and irrelevant noise in RAG answers.
Full Explanation
Chunk size determines how source documents are segmented before being embedded and indexed, which controls how much context lives in each retrievable unit; chunks that are too small can split a single fact across two separate chunks, causing retrieval to miss relevant details that only make sense together, while chunks that are too large can dilute relevance and pull in unrelated content alongside what's needed. Top-k controls how many of the highest-scoring chunks are retrieved per query, so raising or lowering it changes how much surrounding context, relevant or irrelevant, gets passed to the generative step. Adjusting both together directly addresses the scenario's two symptoms: missed details from over-fragmented chunks and irrelevant content from over-broad retrieval. The option describing chunk size as purely a disk-indexing detail unrelated to result count misses that chunk boundaries directly determine what content exists to be retrieved and ranked, which is inseparable from retrieval quality. The option limiting their effect to response style misattributes a retrieval-stage parameter to the generation stage; chunking and top-k happen before generation and shape what the generator even sees. The option calling them FM training hyperparameters conflates retrieval configuration with model training, which are separate stages of a RAG pipeline; neither parameter touches how the foundation model itself was trained. Scope note: optimal chunk size and top-k values typically need empirical tuning per document type and query pattern. Operational check: evaluate retrieval accuracy on a sample of known schedule queries after each chunk-size or top-k change to confirm relevance actually improved rather than just shifted.