A city sets alerting thresholds on its Observability dashboards so that a sustained rise in tool-call error rate or response latency for the permitting agent automatically pages the on-call engineer. What production-monitoring practice does this represent?
Select an answer to reveal the explanation.
Short Explanation
Waiting for residents to call in and complain is the slow way to learn something's wrong. Setting a threshold on error rate or latency and having it page someone the moment things drift is the fast way. That's alerting doing its job before a bad day turns into a bad week.
Full Explanation
Reliability-metric alerting defines thresholds on operational signals — tool-call error rate, latency, and similar metrics — so that a sustained breach automatically notifies the on-call engineer, catching degradation early and shrinking the time between a problem starting and someone responding to it, a core piece of production monitoring for an agentic workload. Continuous evaluation compares agent outputs against a golden dataset to catch quality regressions, a different signal source than live operational metrics like error rate and latency. Canary rollout validation is a deployment-time comparison between two versions on a slice of traffic, not an ongoing alerting mechanism. A principal access boundary policy governs what resources an agent can touch, unrelated to detecting elevated error rates or slow responses. Scope caveat: thresholds set too tight generate alert fatigue and get ignored, while thresholds set too loose miss real degradation, so they need periodic tuning against observed baseline behavior. Operational check: review recent alert history to confirm thresholds have fired for genuine incidents and not chronically for normal traffic variation.