A housing authority runs its eligibility-screening agent's evaluation suite across demographic subgroups of applicants before deployment, rather than checking only an aggregate accuracy score across all applicants combined. What can subgroup evaluation surface that an aggregate score alone would hide?
Select an answer to reveal the explanation.
Short Explanation
An aggregate score is like an average test grade for a whole classroom — a solid class average can still hide a handful of students failing badly. Subgroup evaluation is grading each row of desks separately, which is the only way to see whether one group is quietly getting the short end.
Full Explanation
Aggregate accuracy averages performance across every applicant, so a strong majority subgroup can mathematically offset a weaker minority subgroup and still land on an acceptable-looking overall number, which is exactly the disparity that matters most in an eligibility-screening context where a systematic gap could translate into unequal access to housing assistance. Breaking the same evaluation run down by subgroup exposes that gap directly, because each subgroup's score is reported on its own rather than blended into one figure. Response latency during peak periods is a performance concern unrelated to demographic outcome parity and would show up the same way whether or not the score is segmented. Retrieval-index correctness is a pipeline configuration check, not something subgroup segmentation of outcomes is designed to reveal. Whether system instructions changed since the last deployment is a deployment-metadata question, not a signal that emerges from slicing accuracy by subgroup. Scope caveat: subgroup evaluation is only as good as the subgroup labels available in the golden dataset, so a dataset without demographic coverage can't support this analysis at all. Operational check: pull the per-subgroup accuracy breakdown from the evaluation report and confirm no subgroup's score falls meaningfully below the aggregate before approving deployment.