Week-over-week dashboards show Spanish-language task scores lagging English. What should longitudinal eval analysis drive?
Select an answer to reveal the explanation.
Short Explanation
If Spanish scores keep trailing, that’s not a rounding error—it’s a fairness gap. Grow the bilingual eval set and fix what lags. Hiding the split or cutting Spanish just games the average.
Full Explanation
Longitudinal eval analysis should guide equity-focused municipal improvements when language cohorts diverge. Week-over-week dashboards showing Spanish-language task scores lagging English should drive expansion of bilingual eval sets and prioritized fixes where Spanish lags—retrieval coverage in Spanish sources, prompt language quality, translation-aware routing, and specialist handoffs—not cosmetic averaging tricks. Sustained dashboards make access gaps visible and actionable over time for language-access commitments.
Dropping Spanish support to raise the blended average score manufactures a better number by excluding residents who need service. Hiding language breakdowns so equity gaps are less visible is a governance failure that blocks accountability. Tuning only English prompts because most tickets are “assumed English” ignores measured need and can violate municipal language-access policy.
Exam caveat: longitudinal, language-disaggregated eval is meant to prioritize lagging cohorts—do not hide or drop languages to improve averages. Operational check: enlarge Spanish gold sets, ship targeted retrieval and prompt fixes, re-plot week-over-week Spanish versus English scores, set an equity gate that blocks promotion if the gap widens beyond policy, and report cohort metrics alongside blended scores to leadership.