How does Constitutional AI relate to RLHF in Anthropic's approach?

Question

Accepted Answer

Constitutional AI uses a set of principles (a 'constitution') to critique and revise model outputs, reducing some reliance on human preference labels versus pure RLHF — CAI is RLHF with a written constitution — principles guide self-critique so you need fewer human preference labels for safety.

Answer

Constitutional AI replaces all human feedback and needs zero human input ever

Answer

RLHF and Constitutional AI are identical marketing terms

Answer

Constitutional AI only applies to image models

How does Constitutional AI relate to RLHF in Anthropic's approach?

Short Explanation and Infographic

Full Explanation