A pilot has been running in production for several months. Which practice keeps fairness evaluation meaningful over that time rather than just at launch?
Select an answer to reveal the explanation.
Short Explanation
You can't treat fairness like a launch-day checkbox. You need to keep checking outcomes for relevant groups because the people you serve and the model's behavior can drift over time.
Full Explanation
The correct practice is re-evaluating outcomes for relevant groups on an ongoing basis, since population mix and model behaviour can drift after launch. Input distributions, user populations, and downstream conditions can shift after deployment, so a fairness assessment taken once at launch can become stale; ongoing group-level re-evaluation is what catches drift.
Relying on the pre-launch fairness sign-off is incorrect because model behaviour toward groups is not fixed; changes in the served population or in feature relationships can shift outcomes even with no model changes.
Re-running the pre-launch aggregate accuracy test on a monthly schedule is incorrect because repeating the aggregate metric on a schedule still hides group-level differences the same way it did at launch.
Reviewing user complaint tickets only when their volume triggers an unrelated operational alert is incorrect because tying the review to an unrelated alert threshold means a real but quiet disparity can go unnoticed indefinitely.
For the exam, expect fairness evaluation to be an ongoing monitoring activity, not a one-time gate.