A platform squad wants evaluations that isolate failures across memory, knowledge retrieval, tools, and prompts rather than only scoring final answers. Which evaluation design best supports root-cause localization?
Select an answer to reveal the explanation.
Short Explanation
Correct answer: D. Final-answer scores alone hide where the system broke. Build component evals—memory retention, retrieval quality, tool contracts and error paths, prompt regressions—and still keep end-to-end task success. Wall-clock only (A) and HTTP 200 (B) ignore agent quality. One thumbs-up after demos (C) is not a design. When e2e fails, component scores tell you whether to fix memory compaction, chunking, MCP tools, or prompts first.
Full Explanation
Evaluations for multi-agent solutions should cover memory, knowledge, tools, and prompts—not only overall outcomes. Option D decomposes the system into testable components while retaining end-to-end metrics, enabling root-cause localization. Options A and B are infrastructure or availability signals. Option C is informal and non-repeatable. A robust Foundry-aligned strategy versions each evaluation suite, runs them in CI on changes to prompts/tools/indexes, and attaches failures to the responsible component owner.