A housing AI audit samples human overrides but never samples automated approvals that never see a human. What population flaw should the auditor correct?

Redefining the population to include AI-driven automated approvals—not only human overrides—works because silent auto-approvals often dominate housing decision volume and carry the model’s full effect without human friction. Sampling only overrides biases evidence toward exceptional paths and never tests the machine-only route residents actually experience. Complete populations let auditors assess whether automated approvals meet policy, fairness, and documentation expectations.

Stratifying the override sample by approval and denial outcomes fails because balancing outcome mix among human overrides does not add the automated approvals that never reach a human. Extending sampling to supervisor-reviewed overrides with documented rationales fails because those remain human-reviewed cases and still omit the automated approvals that bypass human review entirely. Rebalancing sampling weights toward recent overrides across regional housing offices fails because reweighting overrides by region still leaves automated approvals invisible; regional coverage of overrides is not the same as including the machine-only route.

Exam caveat: automated-approval logs must be complete and attributable—if logging is missing, the finding may be a logging control failure before sampling can proceed. Operational check: rebuild the housing AI sampling frame from system logs of auto-approvals and overrides, document inclusion rules, sample both paths, and compare exception rates and outcomes so automated decisions are not invisible to the audit.