After a prompt change to the triage agent, automated offline scores looked fine on a tiny smoke set, but production customer satisfaction for multi-agent resolutions dropped 12% week over week. On-call had no alert because only infrastructure uptime was monitored. What monitoring capability should be added?
Select an answer to reveal the explanation.
Short Explanation
Infra can stay green while quality goes red, and that's a common multi-agent failure mode. You need behavioral drift and quality regression monitoring—task success, groundedness, eval scores—with alerts, not just uptime. Coffee machines and code line counts won't save you when production answer quality drops.
Full Explanation
Correct capability: behavioral drift and quality regression detection on multi-agent outcomes, including eval scores, groundedness, and task success, with alerts when production quality degrades beyond thresholds. Monitoring strategies for multi-agent solutions include cross-agent coordination tracking, behavioral drift and quality regression detection, and automated remediation for agent failure patterns. Quality can degrade while infrastructure remains healthy, so product metrics and evaluation signals must be first-class monitors.
Monitoring only virtual machine host uptime and ignoring answer quality metrics is incorrect because uptime alone misses semantic regressions from prompt, model, or tool changes. Paging on-call exclusively when the office coffee machine is empty is incorrect because unrelated facility signals do not detect agent quality drift. Tracking only Python lines of code as a quality signal is incorrect because source line count is not a production quality metric for agent outcomes.
Exam focus: multi-agent monitoring must include quality regression, not only platform availability.