Evaluation shows retrieval misses concentrated on alley easement documents. What is the best next optimization step?
Select an answer to reveal the explanation.
Short Explanation
If the map keeps missing alley easements, fix that drawer first—don’t rearrange the welcome mat. Eval pointed at that corpus, so improve chunking and indexing there. Global temperature hacks or nuking the whole knowledge base ignore the evidence.
Full Explanation
Analyzing evaluation results should drive targeted civic knowledge optimizations where failures concentrate. When eval shows retrieval misses concentrated on alley easement documents, the best next step is prioritizing chunking and indexing improvements for that corpus first—fixing section boundaries, metadata, and embed coverage so easement queries actually hit the right passages. Evidence-guided optimization spends scarce municipal engineering time on the measured gap instead of diffuse rewrites elsewhere in the stack.
Retraining greeting templates because they are unrelated but easy to edit is busywork that does not fix easement retrieval and wastes the eval signal. Deleting the entire knowledge base because one topic failed destroys working corpora and overreacts to a localized failure mode. Raising temperature globally hoping easement answers appear by chance increases invention risk and does not repair missing or poorly indexed source documents.
Exam caveat: optimization priority follows the failure concentration in eval—fix the alley easement corpus pipeline before unrelated cosmetic changes. Operational check: isolate the easement eval slice, improve chunk boundaries and index coverage for those documents, re-run retrieval hit-rate and answer faithfulness on that slice, confirm unrelated domains did not regress, and only then broaden changes if other topics still lag.