An agency's case labels come from two clerks who often disagree, and the written instruction is vague on borderline files. A model matches one clerk closely. What limits treating those labels as ground truth?
Select an answer to reveal the explanation.
Short Explanation
A quiz key is only as trustworthy as the instructions and as stable as two teachers grading the same paper. Your labels cannot outrun a vague instruction and clerks who often disagree. Matching one clerk does not turn that disagreement into perfect ground truth.
Full Explanation
Ground truth in a labeled file is the product of a written instruction and of people applying it. When the instruction is vague on borderline cases and two clerks often disagree, the labels carry that ambiguity, and a model that matches one clerk has learned one clerk's habits rather than a settled fact. Treating the match as proof of solid ground truth ignores the second clerk. Saying the disagreement limits only the model reverses the dependence: the labels are the target, so conflict in the labels limits how trustworthy the target is. Saying a typed label is exact regardless of the instruction assumes the act of storage creates truth the instruction did not define. Exam caveat: high agreement does not by itself prove the labels are right if both clerks share the same misunderstanding, so the instruction still has to be read. Operational check: double-label a sample, measure how often the clerks agree, and walk the borderline files against the written instruction before those labels are treated as ground truth.