A pilot chatbot's evaluation only measures how long its answers are. What does this evaluation approach fail to catch?
Select an answer to reveal the explanation.
Short Explanation
Length tells you how much the model said. It doesn't tell you whether the answer was grounded, relevant, or safe. You need content-based evaluation to catch harmful or policy-violating responses.
Full Explanation
The correct option is the one about whether answers are grounded, relevant, and free of harmful or policy-violating content. Answer length has no consistent relationship with groundedness, relevance, safety, or compliance; a long answer can be just as ungrounded or harmful as a short one, and evaluating only length misses all of that. The option about correct punctuation is incorrect because punctuation is a minor stylistic detail, not the substantive gap length-based evaluation leaves. The option about responding within an acceptable time window is incorrect because response time is an operational metric, unrelated to content quality. The option about the user interface rendering correctly on mobile devices is incorrect because interface rendering is a front-end concern, separate from evaluating the generated content. For the exam, recognise that surface metrics like length cannot substitute for content-based evaluation.