A GitHub Copilot agent is designed to flag potential security vulnerabilities in pull requests. The team evaluates it solely on the number of flags it produces per PR. After one month, developers are ignoring the agent's feedback entirely. What evaluation misalignment caused this?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Measuring a smoke detector by how many times it beeps tells you nothing about how useful it is — what matters is whether it fires when there's actual smoke. Optimizing for flag count rather than flag accuracy turned the agent into a noisy alarm, not a trusted security advisor.
Full explanation below image
Full Explanation
Evaluation metrics must be aligned with the actual intent behind the agent's design. Misaligned metrics can cause the agent to be optimized for the wrong behavior, leading to outcomes that are technically measurable but practically counterproductive.
Why A is correct: The team's intent was to help developers identify real security issues. The metric used (flags per PR) measures volume, not accuracy. An agent optimized for this metric produces more flags — many of which are false positives — leading developers to distrust and ignore all flags. The correct metric should be precision (percentage of flags that are true vulnerabilities) and recall (percentage of true vulnerabilities that are flagged).
Why B is wrong: Evaluation frequency is about how often you measure, not what you measure. Evaluating too frequently does not cause the metric to be misaligned with intent.
Why C is wrong: Model version affects capability, not metric alignment. Even with the best model, evaluating on flags-per-PR would produce the same misalignment.
Why D is wrong: The problem is not that a quantitative metric was used, but that the wrong quantitative metric was chosen. Precision and recall are quantitative metrics that would be correctly aligned with the agent's intent.