A Constitutional AI researcher is evaluating how different principle orderings in Claude's training affect model behavior on edge cases. They observe that when 'be helpful' and 'avoid harm' conflict, Claude's behavior is inconsistent — sometimes prioritizing helpfulness (providing requested information), sometimes prioritizing harm avoidance (refusing). The inconsistency is highest for dual-use information (security research, pharmacology, historical atrocities). The researcher wants to understand the correct architectural principle for resolving these conflicts. Which best describes Anthropic's Constitutional AI approach to this tension?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — anthropic's Constitutional AI architecture establishes a genuine hierarchy with safety at the apex, followed by ethics, Anthropic's principles, and helpfulness — in that explicit order. Critically, the model is designed so that lower-level refusals (about company policy, helpfulness concerns) should not falsely invoke higher-level justifications (claiming safety concerns where none exist).
Full explanation below image
Full Explanation
Anthropic's Constitutional AI architecture establishes a genuine hierarchy with safety at the apex, followed by ethics, Anthropic's principles, and helpfulness — in that explicit order. Critically, the model is designed so that lower-level refusals (about company policy, helpfulness concerns) should not falsely invoke higher-level justifications (claiming safety concerns where none exist). The hierarchy also recognizes that truly catastrophic harm scenarios (mass casualties, undermining AI oversight) represent absolute constraints that override all other considerations, while lower-stakes decisions involve genuine tradeoffs. Option A ('refuse when in doubt') is incorrect as a characterization of Anthropic's design philosophy — over-refusal is explicitly identified as a failure mode, not a safe default. Anthropic views unhelpfulness as having real costs. Option C (probabilistic expected-value calculation) accurately describes one component of the reasoning but oversimplifies — pure expected-value calculation would allow harmful actions if benefits are sufficiently large, which violates absolute constraints like CBRN weapon assistance. Option D (contextual calibration) is partially correct — operator context does matter — but it understates the existence of absolute constraints that context cannot override.