Priya is an AI developer at Contoso Financial Services who has built a Copilot Studio agent that answers questions about investment products. Her manager asks her to set up a formal evaluation framework using a test set before the agent goes live. Priya needs to understand what a properly structured test set must contain to enable accurate AI evaluation scoring in Copilot Studio. Which combination of elements is required in each test set entry for evaluation to function correctly?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of a test set like an answer key for an exam — you need the question (user query), the correct answer (expected response/ground truth), and the textbook chapter it came from (the knowledge context). Without all three, the AI scoring engine can't determine whether the agent's answer was accurate or grounded. Correct answer: B.
Full explanation below image
Full Explanation
A test set in Copilot Studio's AI evaluation framework serves as the authoritative benchmark against which agent responses are measured. Each entry in a valid test set must include three core components: the user query (what the user asks), the expected response — also called ground truth — (what a correct answer looks like), and the relevant context or knowledge source that the agent should draw from.
Option A is incorrect because confidence score thresholds are an output metric, not an input to a test set. You configure threshold tolerances separately in the evaluation settings — they do not form part of the test data itself. Additionally, using the agent's own live response as part of the test set input would create circular evaluation logic.
Option C is incorrect because topic labels are organizational metadata used for routing within Copilot Studio topics — they are not an evaluation input. A fallback response is a safety net for failed interactions, not a test set component.
Option D is incorrect because conversation transcripts may be useful for retrospective analysis, but they are not a structured evaluation input. Star ratings represent human sentiment and are not how Copilot Studio's AI-based evaluation metrics (groundedness, relevance, coherence) are computed — those require expected responses and context, not reviewer ratings.
Option B is correct because the AI evaluation engine in Copilot Studio compares the agent's generated response against the expected response (ground truth) and checks whether the response is grounded in the provided context. Without the expected response, groundedness and relevance scores cannot be computed meaningfully.
Exam tip: Remember the triad — query, ground truth, context. If any one is missing, evaluation scoring breaks down. On the AB-620 exam, watch for options that mix in live agent output or human ratings, which are outputs, not inputs to the test set.