A city SRE team already watches HTTP 5xx rates but lacks insight into GenAI-specific health for the resident assistant. Which monitoring approach best closes that gap?
Select an answer to reveal the explanation.
Short Explanation
Classic 5xx lights won’t tell you the bot started inventing park fees. Add GenAI KPIs—tokens, hallucination proxies, answer ratings—to CloudWatch so civic health includes model behavior.
Full Explanation
City SRE teams that only watch HTTP 5xx miss GenAI-specific health: token burn, quality drift, and resident dissatisfaction can rise while the web tier stays green. Building CloudWatch dashboards that include token usage, hallucination-proxy signals (for example grounding fail rates and policy-block rates), and resident answer ratings alongside classic app metrics closes that gap. Municipal SREs then see cost, quality, and availability in one operational view for the resident assistant rather than assuming model behavior never changes.
Monitoring only web-tier CPU and assuming foundation-model quality never drifts leaves silent failures in answers that residents notice first. Disabling logging of GenAI metrics to reduce dashboard clutter removes the very signals needed for GenAI health. Tracking only the number of PowerPoint slides staff produce each week is unrelated office activity, not system observability for the assistant.
Exam caveat: GenAI observability extends classic SRE metrics with tokens, quality proxies, and user feedback on AIP-C01 monitoring items. Operational check: ship custom metrics and logs for tokens, groundedness or guardrail events, and thumbs-up/down; alarm on sustained quality or token anomalies; review the combined dashboard in weekly SRE ops with product owners.