A housing authority builds a custom autorater to judge whether an eligibility-screening agent's explanations are policy-compliant, after finding that a generic quality scorer kept rating clearly non-compliant explanations as high quality. What justifies choosing a custom autorater over the generic scorer here?

Generic evaluation scorers are typically trained or configured to judge broad quality signals like coherence, fluency, and relevance, none of which encode a specific regulatory or policy rule set — so an explanation can read smoothly, sound confident, and still violate an eligibility criterion the scorer has no representation of. A custom autorater closes that gap by being built specifically to check the agent's output against the housing authority's actual policy rules, which is what a domain-blind generic scorer structurally cannot do. Claiming a custom autorater is always faster confuses implementation choices with an inherent property; speed depends on how each is built, not on which category it falls into. Claiming it removes the need for human review overstates what any autorater can guarantee — even a well-built one benefits from periodic human validation to catch drift. Restating that the generic scorer misses domain-specific compliance without naming why (its optimization target is fluency, not policy) leaves the actual mechanism unexplained, which is the distinction this question is testing. Scope caveat: a custom autorater is only as good as the policy rules encoded into it, so it needs updating whenever eligibility criteria change. Operational check: feed the autorater a known non-compliant explanation and confirm it flags the violation before trusting its scores on live traffic.