A fire district's responsible AI evaluation shows disparate error rates across neighborhoods for a candidate dispatch-aid model. What should the promotion decision be?

Rejecting promotion until disparate error rates across neighborhoods are investigated and addressed is the correct decision when responsible AI evaluation flags a fire-district dispatch-aid candidate. Disparate errors mean the model’s mistakes are unevenly distributed; promoting on average accuracy alone can encode inequitable service risk. Azure ML / responsible AI workflows treat such findings as promotion blockers until root causes (data coverage, label quality, feature bias, thresholding) are understood and mitigated or formally accepted with documented residual risk—acceptance that is rarely appropriate for emergency dispatch aids without explicit oversight.

Rejecting promotion until neighborhood gaps are documented in the model card fails because recording the disparity in a model card does not investigate or address it; documentation alone would allow promotion without mitigating uneven neighborhood harm. Rejecting promotion until post-deployment monitoring for neighborhood drift is set fails because setting up monitoring does not investigate or address the known disparity before promotion; monitoring watches for future drift and is not a substitute for investigation and remediation. Rejecting promotion until global threshold retuning restores overall accuracy levels fails because a global threshold fix can raise average accuracy while leaving or worsening uneven neighborhood error rates, and it allows promotion without understanding and addressing the disparity itself.

Exam caveat: investigation may lead to data fixes, model changes, or a decision not to use ML for that decision point—do not assume a quick hyperparameter tweak always resolves disparity. Involve operational and community stakeholders in defining acceptable error bands. Operational check: hold the promotion gate, analyze error slices by neighborhood, remediate or redesign, re-run responsible AI evaluation, and only then reconsider registry promotion for the dispatch-aid model.