An AI system for public benefits eligibility determination is suspected of denying benefits to eligible applicants at a higher rate than similar manual processes. Which AI audit procedure BEST assesses this concern?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because comparing AI denial rates to manual baselines across demographic subgroups directly tests the hypothesis that the AI is performing differently from the established process. Case-by-case review of AI denials provides evidence of whether individual denials were correct or erroneous.
Full explanation below image
Full Explanation
B is correct because comparing AI denial rates to manual baselines across demographic subgroups directly tests the hypothesis that the AI is performing differently from the established process. Case-by-case review of AI denials provides evidence of whether individual denials were correct or erroneous. Code review (A) can find implementation bugs but won't reveal emergent discriminatory patterns in model outputs. Performance testing (C) assesses operational capacity, not decision accuracy. Privacy compliance (D) is a separate concern from decision accuracy.