A 311 GenAI test uses only auditor-written prompts that do not reflect real resident phrasing. What testing weakness should the auditor address?
Select an answer to reveal the explanation.
Short Explanation
You can't fix the mismatch by expanding auditor-written edge cases, since those still won't sound like real residents. Representative resident phrasing belongs in GenAI test cases. Checking peak latency or verifying municipal citations won't solve the prompt problem either.
Full Explanation
Selecting representative test cases that mirror actual resident language and scenarios works because 311 GenAI behavior on polished auditor prose can diverge sharply from messy real requests—“my trash missed again,” code-switching, typos, or multi-issue rants. Non-representative prompts understate failure modes in routing, tone, unsafe advice, and policy hallucination. Representative cases improve external validity of civic GenAI test conclusions.
Expanding the auditor-written prompt set to increase coverage of edge cases fails because adding more auditor-authored prompts still does not mirror actual resident language and scenarios; the weakness is prompt representativeness, not edge-case volume. Measuring model latency and availability during peak resident call volumes fails because the identified weakness concerns whether prompts reflect real resident language, not system performance under load. Verifying response citations against current municipal code sources for improved accuracy fails because it addresses output citation quality rather than the input-prompt representativeness problem described in the scenario.
Exam caveat: representative testing should include both common and high-harm scenarios; pure popularity sampling can still miss rare dangerous asks. Operational check: build a prompt library from redacted real 311 tickets and call notes (privacy-sanitized), stratify by service type and risk, run the GenAI against that set, and retain the case library as reusable audit evidence for future cycles.