A public library chatbot can be jailbroken into giving harmful DIY chemical advice scraped from open forums. What threat scenario should the auditor evaluate?
Select an answer to reveal the explanation.
Short Explanation
If residents can jailbreak a library bot into dangerous how-tos, misuse testing belongs on the threat list. Public civic GenAI needs guardrails against that path. Privacy leaks about patron borrowing aren't the risk here because they don't involve a jailbroken bot giving out dangerous chemical how-tos.
Full Explanation
Evaluating misuse and jailbreak scenarios that coax public civic GenAI into harmful guidance emphasizes safety threat modeling for library chatbots. The correct scenario works because adversaries can override intended civic-use policies and elicit dangerous DIY advice, especially when the assistant can draw on broad open-forum knowledge. Public municipal GenAI needs misuse cases in the threat model and corresponding guardrails and safety tests—not only benign FAQ coverage.
Privacy leakage scenarios exposing patron borrowing histories through prompts fail as the threat scenario because patron data exposure does not describe jailbreak-driven harmful guidance. Copyright reproduction scenarios where summaries repeat protected book passages verbatim fail conceptually: copyright infringement is unrelated to GenAI misuse that yields dangerous DIY chemical advice. Hallucinated catalog scenarios generating inaccurate summaries of holdings and hours fail because catalog inaccuracies do not address jailbreaks that produce dangerous chemical advice.
Exam caveat: filter vendors and prompt policies can fail under multi-turn social engineering even when single-turn jailbreaks are blocked. Operational check: include harmful-guidance jailbreak cases in library red-team suites, enforce layered refusals and topic blocks, monitor escalations, and require safety sign-off before exposing the public assistant to open-domain knowledge.