English and Spanish housing datasets show unequal agent success. How should performance be compared?
Select an answer to reveal the explanation.
Short Explanation
If Spanish housing chats fail more often, that gap needs its own scoreboard—not a blended average that hides it. Stratified comparison is how you spot equity problems. A language toggle doesn’t prove equal quality.
Full Explanation
English and Spanish housing datasets with unequal agent success require stratified metrics by language so civic equity gaps surface. Comparing performance across language datasets—not only a blended average—shows whether Spanish-speaking residents receive equal housing guidance quality. Equity-aware evaluation is part of professional municipal agent assessment: overall averages can hide systematic underperformance for a language community.
Reporting only the combined average so language gaps disappear conceals inequity and blocks remediation. Dropping Spanish evaluation because English scores are already strong abandons a served population and biases the benchmark toward the stronger language. Assuming unequal success is impossible if the UI has a language toggle confuses interface availability with equal answer quality.
Exam caveat: stratification should cover language and, where relevant, other civic slices (channel, topic, jurisdiction) without using averages as a substitute for subgroup metrics. Operational check: score housing dialogues separately on English and Spanish gold sets, publish both metrics, investigate Spanish failures (retrieval coverage, translation, policy phrasing), and hold release if the equity gap exceeds the city’s agreed tolerance.