A lending-eligibility model is due for production rollout, and the sign-off criteria must catch a fairness gap before it reaches customers. What should the evaluation plan include?
Select an answer to reveal the explanation.
Short Explanation
One number can hide a group getting a much worse deal. You only find that by looking group by group.
Full Explanation
Fairness is a claim about how outcomes are distributed across groups, so it can only be checked by disaggregating results for protected or otherwise relevant groups wherever a fairness risk plausibly exists, alongside the overall performance figures, not by reading a single summary statistic.
The single overall accuracy figure is wrong because an aggregate figure can stay high even while one subgroup is systematically disadvantaged; averaging is exactly what hides the gap.
The performance report scoped to the largest applicant segment is wrong because it excludes the smaller groups most likely to show a disparity, which defeats the purpose of the check.
The sign-off from the model's developers confirming no group was deliberately targeted is wrong because intent is not evidence of outcome; a model can produce disparate results with no one having intended that.
For the exam, expect fairness evaluation to mean group-level outcome analysis, not a developer statement or an aggregate metric.