Before promoting a new multi-agent configuration in Microsoft Foundry, the responsible AI lead requires structured human review of sampled traces: correctness, tool appropriateness, and policy adherence—not only automated scores. Which evaluation practice should the team implement?
Select an answer to reveal the explanation.
Short Explanation
You need a real human review process in Foundry: sample runs, use explicit rubrics, and record reviewer decisions. Automated scores alone won't catch nuanced tool misuse or policy problems. Skipping review, informal skims, and model self-praise aren't review programs.
Full Explanation
The correct practice is to design and implement a human review process in Foundry to evaluate solution behavior on sampled runs with explicit rubrics and recorded reviewer decisions. Evaluation strategies for multi-agent solutions include structured human review, and rubrics plus recorded outcomes complement automated judges and catch issues scores miss. Skipping human review whenever automated BLEU-like scores exceed an arbitrary threshold is incorrect because automated n-gram or similar scores are insufficient alone for tool-using multi-agent policy compliance. Asking an intern to skim one happy-path chat without rubrics or record-keeping is incorrect because informal, unrecorded review lacks rigor and auditability. Treating model self-praise in the final answer as sufficient human review is incorrect because the model’s own confidence language is not an independent human evaluation. Exam focus: human review in Foundry is an explicit evaluation skill, not optional theater.