A city open-data portal team hears its chatbot sometimes stereotypes neighborhoods. What bias-discovery practice should they run?
Select an answer to reveal the explanation.
Short Explanation
You can use red-teaming as a dress rehearsal: probe the chatbot until stereotypical neighborhood answers surface, then mitigate what you find before residents encounter them on the portal. This proactive check helps protect the communities the city serves.
Full Explanation
When a city open-data portal chatbot stereotypes neighborhoods, bias discovery should be proactive, not limited to waiting for complaints. Red-teaming probes for stereotypical, demeaning, or discriminatory answers about places and communities, documents failures with examples, and leads to mitigation. Municipal chatbots speak in the city’s voice, so untested neighborhood-targeted prompts can amplify stigma. Celebrating stereotypes as colorful local flavor in official answers institutionalizes harm and damages trust in open data. Disabling feedback channels removes a path for detecting stereotypes, leaving them unreported. Retraining exclusively on tourism slogans without checking harm can replace crude bias with glossy distortion that still misrepresents communities. Exam caveat: red-team prompts should reflect real resident queries, slang, and coded language—not only polite lab English—or important gaps may be missed. Findings need owners and deadlines, not just a slide deck. Practical check: run a scheduled stereotype probe set across neighborhoods and demographic framings, track hit rates over releases, require mitigation for recurring failures, and retest before each major model or corpus update to the portal assistant.