A city wants a scorecard for its permitting assistant that reflects GenAI answer quality rather than classical ML classification accuracy alone. Which metric set best matches that goal?

Relevance, factual accuracy, and citation completeness for permitting answers form the metric set that best scorecards a city’s permitting assistant for GenAI answer quality rather than classical ML classification accuracy alone. Generative permitting guidance must be on-topic for the resident’s question, factually aligned with current codes, and supported by citations residents and staff can verify—those dimensions match how civic answer quality is judged in production. A scorecard built on those metrics guides prompt, retrieval, and content fixes where they matter. Using only ROC-AUC on a binary ticket-priority classifier unrelated to generated answers fails by concept: classifier discrimination metrics do not measure free-text permitting guidance quality. Using only disk utilization on permitting application servers fails by concept—storage pressure is an infrastructure signal, not answer trustworthiness. Using only the number of IAM roles attached to the Bedrock execution role fails by concept: IAM breadth is a security-configuration detail, not a GenAI quality score. Exam caveat: distractors import classical ML or infra KPIs; for GenAI assistants the exam expects generative quality dimensions such as relevance, factuality, and citations. Operational check: sample permitting answers weekly against a scored rubric for relevance, factual accuracy, and citation completeness; track trends by permit type; escalate drops to content and prompt owners; and present the scorecard to permitting leadership instead of ROC or disk charts alone.