Housing caseworkers report that an AI eligibility screener shows higher denial rates in some neighborhoods after go-live. What concern should the AI ethics review prioritize?
Select an answer to reveal the explanation.
Short Explanation
If one neighborhood keeps getting more denials from the same screener, that is a fairness alarm—not a “good enough on average” shrug. Benefits access is where AI ethics meets real families, and disparate impact belongs on the review agenda first.
Full Explanation
When housing caseworkers report that an AI eligibility screener shows higher denial rates in some neighborhoods after go-live, the AI ethics review should prioritize societal fairness risk that the screener may produce disparate impacts on municipal benefit access. Neighborhood-correlated denial gaps are a civil and trust issue even when citywide average accuracy looks strong.
Why the other options fail by concept: Declaring denial-rate gaps irrelevant if average accuracy looks high citywide privileges aggregate metrics over distributional harm. Telling caseworkers to ignore disparities until a lawsuit arrives is reactive and unethical public administration. Assuming neighborhood differences always prove the model is working as intended confuses disparity with validity without investigation.
Exam caveat: Disparate impact findings require careful analysis—proxy features, historical label bias, and unequal data quality can all drive gaps. Fairness review is not automatic model abandonment; it is evidence-gathering, mitigation, and possible rollback with human appeals intact.
Operational check: Slice denial, approval, and appeal-overturn rates by neighborhood and protected-class proxies permitted by law; compare to pre-AI baselines; pause or constrain automated denials if gaps exceed policy thresholds; and document mitigations (feature review, threshold changes, mandatory human second look) before continuing the screener.