After a prompt change to the triage agent, automated offline scores look stable, but live users report more misroutes. What monitoring gap should you close?
Select an answer to reveal the explanation.
Short Explanation
D closes the gap. Offline evals can miss live distribution shift; quality regression and behavioral drift detection on production signals—misroute rate, human corrections—tied to release versions catch what offline suites miss. A trusting offline only is the failure mode in the stem. B CPU is not routing quality. C cutting traffic is not monitoring. Pair offline gates with online quality monitors for multi-agent releases.
Full Explanation
Correct Answer — D
Behavioral drift and quality regression detection must include production outcome metrics (for example misroutes and corrections) correlated to versions, complementing offline evaluation scores.
Why A is wrong: Offline stability can mask live misrouting.
Why B is wrong: Infrastructure CPU does not measure routing quality.
Why C is wrong: Stopping traffic avoids learning rather than monitoring.
Exam tip: Online quality monitors catch regressions offline sets miss.