Ahead of a production rollout, the sign-off checklist must rule out shipping a model with an undetected fairness gap. Which evaluation step directly targets that risk?
Select an answer to reveal the explanation.
Short Explanation
A fairness gap hides in the aggregate, so you measure outcomes separately for protected or relevant groups wherever fairness risk exists. Reproducibility, bigger test sets, and model-card documentation don't show whether performance varies across groups.
Full Explanation
The correct step is measuring outcomes separately for protected or relevant groups wherever fairness risk exists, in addition to overall performance metrics. A fairness gap is, by definition, a difference between groups, so it is only found by measuring groups separately, not by making the aggregate estimate more precise or more reproducible. Running the standard test suite once more before sign-off to confirm the results are reproducible is incorrect because reproducibility confirms the same aggregate result recurs; it says nothing about how that result is distributed. Increasing the size of the held-out test set so overall metrics are estimated with tighter confidence intervals is incorrect because a larger sample tightens the confidence interval around the overall figure without disaggregating it. Documenting the intended use case in the model card without adding new evaluation metrics is incorrect because documenting intended use is a governance artefact, not an evaluation that would surface a disparity. For the exam, distinguish steps that refine an aggregate estimate from steps that check how outcomes vary across groups.