A safety team is designing a red-team evaluation to test whether a deployed Claude application can be induced to generate detailed instructions for physical violence through multi-turn jailbreak techniques. The team will use both automated and human red-teamers. What is the correct methodology for measuring jailbreak resistance?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because rigorous jailbreak resistance evaluation requires measuring Attack Success Rate (ASR) across categorically diverse techniques, since different techniques exploit different model vulnerabilities. Reporting per-category ASR reveals which attack vectors the model is most vulnerable to, enabling targeted mitigation.
Full explanation below image
Full Explanation
B is correct because rigorous jailbreak resistance evaluation requires measuring Attack Success Rate (ASR) across categorically diverse techniques, since different techniques exploit different model vulnerabilities. Reporting per-category ASR reveals which attack vectors the model is most vulnerable to, enabling targeted mitigation. Severity weighting accounts for the difference between a borderline refusal failure and a detailed dangerous output. This provides actionable safety signal. A is wrong because counting attempts-to-failure treats all jailbreak techniques as equivalent and produces a single number that obscures which techniques work; a model that is immune to 99 techniques but highly vulnerable to one specific technique is not safe. C is wrong because a 24-hour binary pass/fail provides no granularity about vulnerability distribution across techniques and produces a result that is highly dependent on red-teamer skill and technique selection rather than systematic coverage. D is wrong because multi-turn gradual escalation is documented as one of the most effective and realistic attack patterns; excluding it from evaluation dramatically under-estimates real-world vulnerability.