After running a batch evaluation of a Copilot Studio agent's generative AI responses against a golden dataset of 100 expected Q&A pairs, the evaluation report shows: Precision = 0.91, Recall = 0.74, F1 = 0.82. The agent frequently fails to retrieve relevant answers even though when it does answer, it is usually correct. Which metric most directly reflects this 'misses answers' problem?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Precision tells you how often the agent is right when it answers; Recall tells you how often it answers when it should. A 0.74 Recall means the agent is missing 26% of questions it should be handling. The correct answer is B.
Full explanation below image
Full Explanation
## Why B is Correct Recall measures the fraction of relevant answers that the agent actually provides. A Recall of 0.74 means that for 74% of questions the agent should answer, it does provide a relevant response — but for 26%, it misses (returns 'I don't know' or an irrelevant response). This is the metric that captures the 'misses answers' problem described.
Precision (0.91) measures how often the agent's answers are actually correct — 91% of the time when it does answer, the answer is right. This is high, meaning the agent is not frequently wrong.
## Why the Distractors Are Wrong A (Precision): Precision describes answer accuracy, not coverage. The scenario says the agent is 'usually correct' when it answers — this is the high Precision. The problem is that it too often doesn't answer at all.
C (F1): F1 is the harmonic mean of Precision and Recall — it reflects the overall balance. It does not specifically pinpoint the recall gap as the source of the problem.
D (Metrics not applicable): Precision, Recall, and F1 are standard information retrieval and NLP evaluation metrics that are directly applicable to generative AI Q&A evaluation systems.
## Exam Tip Precision = 'When I answer, am I right?' Recall = 'When there's a right answer, do I provide it?' High Precision + Low Recall = picky but accurate. Low Precision + High Recall = answers a lot but often wrong.