A 311 RAG assistant is never tested for hallucination against a gold-answer set. What should the auditor recommend?
Select an answer to reveal the explanation.
Short Explanation
RAG still hallucinates — test it against gold answers like a pop quiz with an answer key. You can't treat retrieval precision and citation coverage as a factuality lab. You can't use fluency and satisfaction scores as hallucination control.
Full Explanation
A 311 RAG assistant can still hallucinate ordinances, service hours, or procedures even when retrieval is present. Groundedness and factuality testing against a gold-answer set measures whether answers stick to approved municipal sources rather than invented civic facts. The auditor should recommend systematic factuality and groundedness testing before and after production changes so residents are not steered by fabricated guidance about city services.
Testing only that the RAG assistant cites sources confirms a citation exists, not that the cited or generated content is actually correct; an answer can cite a real ordinance and still misstate what it says, which is exactly what groundedness testing against gold answers is meant to catch. Tracking retrieval precision and citation coverage fails as a substitute for factuality testing because precision measures retrieval relevance, not whether the generated answer is factually correct or grounded in approved sources. Evaluating fluency and resident satisfaction scores during pilot fails as hallucination control because fluent, likable answers can still invent civic facts, and user satisfaction does not verify factuality.
Exam caveat: GenAI civic assistants need groundedness and factuality techniques explicitly; RAG is not a free pass on the exam or in operations. Operational check: maintain a gold-answer suite for top 311 intents, run factuality tests on each release, and track hallucination defect rates with clear municipal owners and stop-ship criteria.