A 311 evaluation harness must check whether answers stick to the city’s published service catalog. Which quality metric fits best?

Scoring groundedness against the city's published service catalog is the quality metric that fits a 311 evaluation harness checking whether answers stick to official services. Groundedness asks whether claims are supported by catalog passages (service names, eligibility, SLAs) rather than invented programs or timelines. That metric directly targets the civic failure mode of fluent but false 311 guidance and complements relevance and safety scores without replacing them.

Scoring coherence against the city's published service catalog fails because coherence checks logical flow and consistency, not whether service names, eligibility, or SLAs are supported by catalog passages; a response can be coherent while still inventing programs or timelines. Scoring fluency against the city's published service catalog fails because fluency checks smooth, natural wording, not factual support; fluent phrasing can still misstate services, eligibility, or SLAs from the catalog, which is exactly the hallucination risk groundedness scoring is meant to catch. Scoring relevance against the city's published service catalog fails because relevance checks whether an answer is on-topic for the request, not whether its claims are supported by catalog passages; a relevant answer can still cite non-existent services or wrong timelines.

Exam caveat: Groundedness requires the evaluation harness to supply or retrieve the same catalog the bot uses; mismatched corpora produce false failures or false passes. Catalog updates must trigger eval corpus updates so deprecated services are not still treated as correct.

Operational check: Wire groundedness scoring to the published service catalog version pinned in the harness; include items that tempt hallucination of non-existent city services; gate releases on groundedness thresholds; review ungrounded outputs with 311 operations; and alert when catalog publish events are not followed by eval refresh.