An organization's AI red team discovers that a deployed LLM can be made to ignore its system prompt and generate harmful outputs by prefixing requests with a specific multi-token sequence. What risk management response is MOST appropriate?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — one discovered jailbreak that lets the model ignore its system prompt and produce harmful outputs is an active safety vulnerability. You should take it offline immediately and report it through formal incident management.
Full Explanation
Immediately taking the model offline and reporting the vulnerability through the organization's AI incident management process is correct. A discovered jailbreak that allows harmful outputs is an active safety vulnerability requiring immediate response; the model should be taken offline to prevent exploitation, and the incident should go through formal AI incident management including executive notification, root cause analysis, and remediation. Documenting the finding for the next quarterly model update delays response to an active vulnerability. Informing only the data science team and asking them to retrain the model bypasses formal incident management and executive accountability. Publishing the vulnerability publicly could enable widespread exploitation before a fix is deployed.