You are building an evaluation suite for a multi-agent RAG helper that also calls MCP tools. Stakeholders want coverage beyond final answer text. Which evaluation design best matches the exam’s multi-agent validation scope?
Select an answer to reveal the explanation.
Short Explanation
You need separate evaluations for memory fidelity, knowledge/retrieval quality, tool selection and arguments, and prompt/version behavior, not just final-string similarity. That catches wrong-tool-right-looking-answer failures and memory bleed. Front-door HTTP codes, GPU utilization, and one-time spell-check don't score multi-agent quality.
Full Explanation
The correct design is separate evaluations for memory fidelity, knowledge/retrieval quality, tool selection and arguments, and prompt/version behavior—not only final-string similarity. Design evaluations for memory, knowledge, tools, and prompts. Examples: did short-term memory retain the user preference? Did retrieval return the right chunks? Did the agent pick the refund tool with valid parameters? Did a prompt version regress refusal rates? Counting only HTTP 200 rates from the web app front door is incorrect because availability of the UI is not semantic or tool correctness. Measuring only average GPU utilization is incorrect because infrastructure utilization does not score agent reasoning quality. Spell-checking the system prompt once at authoring time is incorrect because static authoring checks do not replace runtime and regression evaluation. Store evaluation results with correlation to prompt hashes and model deployment versions.