Before promoting a multi-agent customer-support graph from staging to production in Microsoft Foundry, the quality team must score groundedness, tool-choice appropriateness, and tone on a fixed set of scenarios with human reviewers. What should you design first?
Select an answer to reveal the explanation.
Short Explanation
Answer D. AI-500 evaluation skills include designing human review processes in Foundry and evaluations for memory, knowledge, tools, and prompts. Gate release on structured human (and automated) scoring of real scenarios. Prod-only dashboards catch issues too late. Ungated git deploys skip quality. BLEU against web text is not a multi-agent support quality measure.
Full Explanation
Correct answer: D. Design and implement evaluation and validation strategies: human review in Foundry plus targeted evaluations of memory, knowledge, tools, and prompts. Curate scenarios that exercise retrieval quality, correct tool selection, safe refusals, and multi-agent handoffs; capture reviewer scores and block promotion on thresholds.
A is insufficient alone: monitoring is necessary after release but does not replace pre-production evaluation gates.
B is incorrect: continuous delivery still needs automated and human evaluation gates for agent quality and safety.
C is incorrect: classical n-gram metrics on unrelated corpora do not measure groundedness or tool appropriateness for support agents.
Combine human review with LLM-as-a-judge and regression suites for scale.