Two systems are queued for production rollout: an internal document-search tool and a customer-facing insurance-pricing model. Which one most clearly requires group-level fairness evaluation, and why?
Select an answer to reveal the explanation.
Short Explanation
Ask who the outcome lands on. If you're pricing insurance for real people, group-level fairness evaluation is what catches prices that differ by protected characteristic.
Full Explanation
The correct answer is the pricing model, because its outputs materially affect real people's cost of insurance and can plausibly differ by protected characteristic. Evaluation depth should track the risk a system's decisions pose to people, and a pricing decision that affects real customers and could plausibly vary by protected characteristic is a clear case for group-level analysis.
The document-search tool option is incorrect because internal tooling is not automatically held to a higher bar than customer-facing systems; risk to people, not audience, drives the requirement.
The both-equally option is incorrect because a uniform checklist ignores that risk varies by use case, which is the whole basis for prioritising where fairness evaluation matters most.
The neither option is incorrect because disparate impact can arise through correlated proxy variables even when no demographic field is used directly as an input.
For the exam, scope fairness evaluation to where outcome risk to people is highest, not to a fixed list of input fields.