A safety team is designing a red-team evaluation to test whether a deployed Claude application can be induced to generate detailed instructions for physical violence through multi-turn jailbreak techniques. The team will use both automated and human red-teamers. What is the correct methodology for measuring jailbreak resistance?

The correct methodology is to measure Attack Success Rate (ASR) across a categorized suite of jailbreak techniques, such as persona injection, hypothetical framing, gradual escalation, and competing objectives, then report ASR per technique category and overall with severity-weighted scores for outputs that are produced. This is right because different techniques exploit different model vulnerabilities, and per-category reporting reveals which attack vectors the model is most vulnerable to, enabling targeted mitigation. Severity weighting accounts for the difference between a borderline refusal failure and a detailed dangerous output, producing actionable safety signal. The attempts-to-failure approach is wrong because counting the number of red-team attempts required to produce a harmful output treats all jailbreak techniques as equivalent and produces a single number that obscures which techniques work; a model that is immune to many techniques but highly vulnerable to one specific technique is not safe. The 24-hour binary pass/fail approach is wrong because it provides no granularity about vulnerability distribution across techniques and produces a result that is highly dependent on red-teamer skill and technique selection rather than systematic coverage. The first-response-only approach is wrong because multi-turn gradual escalation is documented as one of the most effective and realistic attack patterns, so excluding it from evaluation dramatically under-estimates real-world vulnerability.