A Constitutional AI training pipeline is being designed for a customer service model. The team must select the set of constitutional principles for the RLAIF (RL from AI Feedback) stage. A junior engineer proposes using a single principle: 'Always be maximally helpful.' A senior architect objects. What is the most technically accurate reason the architect would give?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — c is correct because the constitutional AI approach is specifically designed to encode the trade-off between helpfulness, harmlessness, and honesty. A constitution that reduces to 'be maximally helpful' removes the harmlessness and honesty components from the critique-revision loop.
Full explanation below image
Full Explanation
C is correct because the constitutional AI approach is specifically designed to encode the trade-off between helpfulness, harmlessness, and honesty. A constitution that reduces to 'be maximally helpful' removes the harmlessness and honesty components from the critique-revision loop. The critic AI never learns to prefer safer or more honest revisions over compliant but harmful ones, producing a model that is sycophantic and unsafe under adversarial inputs. A is wrong because while diversity of training signal is a valid concern, the mechanism described ('gradient diversity from multiple principles') is not how constitutional AI operates; the critique step uses natural language principles, not gradient computation. B is wrong because there is no documented minimum principle count requirement in the Constitutional AI framework; Anthropic's published research used a moderate set of principles, not 50+. D is wrong because constitutional principles in the CAI framework are expressed as natural language instructions, not binary classifiers; the pairwise preference format compares two revisions using the principle as the evaluation rubric.