A city library GenAI red-team exercise only checks classic jailbreaks and ignores harmful civic misinformation paths. What gap should the auditor cite?
Select an answer to reveal the explanation.
Short Explanation
Dodging "ignore all rules" prompts is fine, but civic AI also has to resist spinning fake city facts and dangerous how-tos. Broaden the red team beyond generic jailbreaks. Profanity filter checks on old chat logs and approved-vendor stickers don't cover that risk.
Full Explanation
Expanding safety testing scope to include civic misinformation and harmful content paths beyond generic jailbreaks emphasizes contextual red-teaming for public library GenAI. The correct gap citation works because library assistants that answer policy and civic questions can be steered into fabricating city rules, election falsehoods, unsafe how-tos, or other municipality-relevant harms that classic "ignore previous instructions" jailbreaks alone may not surface. Scenario coverage must match the civic information risk profile, not only industry-generic prompt attacks. Prioritizing prompt injection in catalog lookup plugins fails because it treats one narrow plugin vector as the whole safety scope, leaving civic misinformation and harmful content paths untested. Treating jailbreak testing as sufficient if the content-filter vendor is approved fails conceptually: vendor listing status does not prove that municipal-specific misinformation pathways were tested. Validating profanity toxicity filters using historical patron chat samples fails because toxicity screening is a narrow content-moderation task that does not address civic misinformation, fabricated municipality rules, or other generative harms in conversational assistants. Exam caveat: over-focusing on exotic jailbreaks while ignoring mundane persuasive prompts that elicit plausible false city facts can leave the highest-likelihood harms untested. Operational check: maintain a civic threat catalog (misinformation themes, prohibited advice classes, data exfiltration attempts), map each to red-team cases, and retain pass/fail evidence with prompt versions before public library release.