A clerk RAG assistant retrieves large ordinance packets, and token spend rises because generation includes lengthy low-value context. Which efficiency practice best reduces tokens while preserving useful citations?
Select an answer to reveal the explanation.
Short Explanation
You do not haul the whole filing cabinet to the counter—just the folders that matter. Prune RAG context to the top citations before generation so the clerk bot spends tokens on signal, not bulk.
Full Explanation
Clerk RAG assistants that retrieve large ordinance packets often waste tokens by stuffing low-value passages into the generation prompt. Pruning retrieval context to the top citations—by score, recency, or section relevance—before generation shrinks the prompt, focuses the model on usable authority, and still preserves the citations clerks need to defend fee or ordinance answers. This is context-window optimization for municipal RAG: spend tokens on signal, not the entire filing cabinet.
Sending the entire unranked corpus into every prompt maximizes cost, can dilute attention, and may hit context limits without improving citation quality. Disabling retrieval and asking the FM to invent fee citations from memory abandons grounding and creates hallucinated legal references—unacceptable for clerk work. Doubling the context window and always filling it to the maximum treats capacity as something to saturate rather than optimize, increasing cost and often degrading focus.
Exam caveat: RAG efficiency items favor selective context / top-k pruning over corpus dumps, invented citations, or “always fill the window.” Operational check: measure prompt tokens and answer faithfulness before/after pruning, retain citation IDs in the UI for clerks, tune top-k with ordinance SMEs, and alert when retrieval returns empty or low-score sets that would force unsafe generation.