Over six weeks, a support multi-agent’s automated evaluation suite shows rising policy-violation rates even though code deploys were frozen. Knowledge articles were updated daily by content owners. What monitoring strategy best detects and explains this pattern?
Select an answer to reveal the explanation.
Short Explanation
Correct answer: D. Code freezes do not freeze quality when knowledge changes under the agents. You need behavioral drift and quality-regression monitoring: versioned eval sets, scores versus a pinned baseline, and correlation of drops with knowledge updates plus model/prompt versions. Jump-box CPU (A), PR counts (B), and AAD sign-in alerts (C) might be useful ops signals but they will not explain rising policy violations after article edits. Treat content releases like model releases—evaluate before and after.
Full Explanation
Multi-agent monitoring must include behavioral drift and quality regression detection, not only infrastructure metrics. With frozen code, daily knowledge updates can still shift retrieval context and agent answers into policy violations. Option D establishes versioned evaluations, baseline comparison, and correlation with knowledge and model/prompt versions—enabling root cause analysis. Options A–C measure infrastructure, SDLC activity, or identity health and do not surface answer-quality drift. Best practice is automated evaluation on each significant knowledge publish, with remediation paths such as rolling back articles, tightening guardrails, or adjusting retrieval filters.