A product team runs a multi-agent claims workflow in Microsoft Foundry. Human spot-checks cannot keep up with nightly regression volume. They need an automated evaluation loop that scores agent final answers for faithfulness to retrieved knowledge and for policy compliance, then flags regressions before promotion. Which approach best meets this continuous-improvement need?

The best approach is the LLM-as-a-judge pipeline that scores each run against rubrics for faithfulness and policy compliance, stores scores with run metadata, and fails the promotion gate when scores drop below thresholds. This implements continuous improvement with an LLM-as-a-judge framework: automated rubrics for faithfulness and policy compliance, persisted scores, and a hard promotion gate on regression. This aligns with Foundry-oriented evaluation practice for multi-agent solutions where volume exceeds pure human review. Replacing human review entirely with exact-string unit tests is incorrect because exact string assertions do not tolerate valid paraphrase and fail to detect semantic hallucinations or policy violations. Increasing the base model temperature on the orchestrator only and re-running a single golden prompt once per week is incorrect because changing temperature and sampling a single prompt weekly does not provide systematic, comparable evaluation across workflows. Disabling tool calling in staging so evaluations measure only pure language generation latency is incorrect because disabling tools evaluates a reduced system that does not represent production tool-using agents. In practice, pair automated judges with human review on stratified samples and store correlation IDs so failed runs can be replayed.