County 311 is switching the bot to a newer multimodal deployment and needs assurance before full cutover. What should the team do?

Assessing quality metrics on the new multimodal deployment before completing cutover is what County 311 should do when switching the bot. Newer multimodal endpoints may change answer style, tool use, retrieval interaction, latency, or failure modes even when demos look polished. Comparing quality metrics—task accuracy, groundedness, safety flags, and latency—against the prior baseline on representative 311 intents reduces the risk that a capability upgrade silently degrades resident service. Assessing latency metrics on the new multimodal deployment before completing cutover captures speed alone and ignores the core requirement: quality metrics on the new multimodal deployment must meet or exceed the 311 baseline before any traffic moves. Assessing cost metrics on the new multimodal deployment before completing cutover captures spend alone and ignores the core requirement: quality metrics on the new multimodal deployment must meet or exceed the 311 baseline before any traffic moves. Assessing throughput metrics on the new multimodal deployment before completing cutover captures capacity alone and ignores the core requirement: quality metrics on the new multimodal deployment must meet or exceed the 311 baseline before any traffic moves. Exam caveat: model and app upgrades in civic generative deployments call for evaluation-before-cutover rather than big-bang switches driven by release notes alone. Operational check: shadow or canary the new multimodal deployment on a scored 311 evaluation set, require metric parity or improvement versus baseline, then ramp traffic; keep rollback to the prior deployment ready until soak metrics hold.