You are building an evaluation suite for a multi-agent RAG helper that also calls MCP tools. Stakeholders want coverage beyond final answer text. Which evaluation design best matches the exam’s multi-agent validation scope?
Select an answer to reveal the explanation.
Short Explanation
Answer A. AI-500 asks you to evaluate memory, knowledge, tools, and prompts as distinct concerns. Final-answer string match alone misses wrong-tool-right-looking-answer failures and memory bleed. Front-door HTTP codes, GPU utilization, and one-time spell-check are not multi-agent quality evaluations.
Full Explanation
Correct answer: A. Design evaluations for memory, knowledge, tools, and prompts. Examples: did short-term memory retain the user preference? Did retrieval return the right chunks? Did the agent pick the refund tool with valid parameters? Did a prompt version regress refusal rates?
B is incorrect: availability of the UI is not semantic or tool correctness.
C is incorrect: infrastructure utilization does not score agent reasoning quality.
D is incorrect: static authoring checks do not replace runtime and regression evaluation.
Store evaluation results with correlation to prompt hashes and model deployment versions.