Housing referral replies may favor some neighborhoods or languages. How should the city audit the LLM for fairness?
Select an answer to reveal the explanation.
Short Explanation
Fairness for housing referrals means checking answers across ZIP codes and languages—not just downtown English. If one neighborhood always gets thinner resource lists, the audit should catch it.
Full Explanation
Housing referral replies may favor some neighborhoods or languages, so the city should audit the LLM for fairness by auditing housing referral replies across ZIP codes and languages for unequal resource suggestions. Fairness here is outcome and allocation oriented: if one ZIP systematically receives thinner shelter, legal-aid, or benefits resource lists—or if non-English prompts get weaker guidance—the assistant amplifies geographic and linguistic inequity. Cross-ZIP and cross-language audits make those gaps measurable instead of assumed away. Auditing only English prompts from a single downtown ZIP code samples the privileged slice and misses the disparity the city fears. Assuming fairness because the model card mentions ethics once confuses documentation claims with empirical municipal evidence. Banning all non-English residents from using the referral chatbot “solves” disparity by exclusion and violates equitable access goals. Exam caveat: define metrics (resource count, specificity, escalation rate, refusal rate) and minimum sample sizes per slice; also separate retrieval corpus gaps from generation bias so remediation targets the right layer. Operational check: build a stratified housing prompt set by ZIP and language, score referral quality per slice, flag statistically or policy-meaningful gaps, and remediate via data, retrieval content, or policy before declaring the housing LLM fair enough for production.