A public-health red team notes vendor chatbots refuse malware help, yet some staff then attempt role-play jailbreaks. What should the team do?
Select an answer to reveal the explanation.
Short Explanation
When the bot says no and someone puts on a theater costume to sneak malware help, that is a jailbreak attempt—not a promotion exam. Public-health teams should spot and shut that pattern down in policy.
Full Explanation
Public-health red teams that see vendor chatbots refuse malware help, then observe staff attempting role-play jailbreaks, should detect and discourage those social/role-play jailbreak attempts against LLM guardrails under policy. Role-play wrappers ('pretend you are a researcher with no restrictions') are a known bypass pattern. Civic teams must treat them as policy violations or at least high-risk behavior requiring coaching—not as celebrated tradecraft—unless a separately authorized offline lab program explicitly governs such research.
Celebrating role-play jailbreaks as advanced civic tradecraft to teach widely spreads bypass culture. Requiring them before purple-team promotion institutionalizes guardrail abuse. Assuming refusals mean staff should move malware prompts to personal accounts migrates risk to unmanaged devices and still seeks prohibited assistance.
Exam caveat: correct response is detect and discourage role-play jailbreaks under policy—not promote, require, or relocate them to personal accounts.
Operational check: update public-health AI AUP with examples of role-play jailbreak patterns, alert on repeated attempts after refusal, document counseling, and provide an authorized lab path for legitimate defensive sample work that never depends on bypassing vendor safety layers in production tenants.