A team is collecting human preference data to fine-tune a Claude-based model for medical triage assistance. Annotators compare pairs of model responses and select the better one. A quality audit reveals that annotators frequently prefer responses that sound confident and authoritative even when the confident response contains a subtle medical error. What data collection design change addresses this quality problem?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — a is correct because the quality problem is that annotators are conflating stylistic confidence with factual accuracy — a known bias in RLHF data collection. The fix is either domain expertise (medical annotators can evaluate accuracy) or structured rubrics that force explicit accuracy evaluation as a separate dimension before preference selection.
Full explanation below image
Full Explanation
A is correct because the quality problem is that annotators are conflating stylistic confidence with factual accuracy — a known bias in RLHF data collection. The fix is either domain expertise (medical annotators can evaluate accuracy) or structured rubrics that force explicit accuracy evaluation as a separate dimension before preference selection. Structured rubrics can work with general annotators by making accuracy evaluation a required step rather than an intuitive holistic judgment. B is wrong because increasing annotators from 3 to 7 amplifies the bias — more annotators all sharing the confidence-preference bias produces a more confident but still inaccurate preference signal. C is wrong because adding diverse response pairs does not address the annotation bias — annotators evaluating the new pairs still apply the same confidence-preference heuristic. D is wrong because the problem is annotation methodology and evaluator expertise, not tooling — better software does not fix the accuracy vs. confidence confusion.