An AI system used by a government agency to assess eligibility for public housing produces systematically lower scores for families with non-English surnames. The agency's AI vendor claims the model does not use ethnicity or national origin as input features. Which investigation approach would BEST determine whether discrimination is occurring?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you need matched-pair testing because it isolates the surname effect while holding financial and household profiles constant. If scores differ systematically, you've got direct evidence of proxy discrimination even when national origin isn't an explicit feature.
Full Explanation
The best approach is an audit using matched pairs — applicants with identical financial and household profiles except for surnames associated with different national origins — and analyzing whether predicted scores differ systematically. This directly tests proxy discrimination by isolating the surname effect, which carries national origin information, while controlling for other variables. Systematic score differences are direct evidence of proxy discrimination even if national origin is not an explicit feature. Reviewing source code for absent ethnicity or national origin fields only shows direct feature use, not indirect proxy effects. Surveying affected families gives anecdotal evidence but cannot establish statistical discrimination. Vendor certification is an attestation with no evidentiary value for the specific discrimination question.