A pilot deployment's model reports 92% overall accuracy, yet a stakeholder still flags a fairness concern. What is the right next step?
Select an answer to reveal the explanation.
Short Explanation
High average accuracy answers different question than fairness concern. You need to break outcomes down by protected or relevant groups before arguing with the number.
Full Explanation
One aggregate accuracy figure can be high while performance for one or more groups is meaningfully worse; disaggregated evaluation is the only way to see that. The correct next step is to evaluate model outcomes broken out by protected or relevant groups, because overall accuracy does not show how error is distributed across those groups.
Accepting 92% overall accuracy as evidence the concern is unfounded is incorrect because treating overall accuracy as proof of fairness is the exact reasoning that lets group-level gaps go undetected.
Re-running the same aggregate accuracy calculation on larger sample is incorrect because larger sample makes same aggregate number more precise without answering group-level question at all.
Asking stakeholder to provide statistical proof of fairness problem before doing further evaluation is incorrect because it shifts burden of evaluation onto person raising concern, when evaluation is governance team's responsibility to perform.
For exam, treat fairness concern as prompt to disaggregate results, not as claim that needs external proof before you will look.