A tool-library FAQ helper is scored on the share of answers a reviewer marks factually correct. Leadership treats that share as the success target. They are not asking for a homework confusion matrix. Which metric is this?
Select an answer to reveal the explanation.
Short Explanation
Think of a reviewer putting a check mark on answers that are actually true. That share is business-facing accuracy. A homework confusion matrix, BLEU on folder colors, and GPU heat do not replace the reviewer's mark.
Full Explanation
Accuracy as a business-facing, human-judged correctness rate is what leadership set: the share of tool-library FAQ answers a reviewer marks factually true. For GenAI helpers where truthfulness is the success target, that operational definition of accuracy is the Domain 2 metric—not a classroom confusion matrix for its own sake. The reviewer’s check mark is the score.
A confusion-matrix homework problem with no reviewer is academic classification machinery without the human judgment asked for, and it assumes discrete labeled classes the FAQ may not have. BLEU on unused folder colors is an irrelevant language-overlap score on metadata that does not equal factual correctness of answers. GPU temperature during review is host telemetry and cannot substitute for a reviewer’s correctness mark.
Exam caveat: human-judged accuracy still needs sampling design and disagreement handling—single-reviewer bias can inflate the rate if hard cases are skipped. Operational check: publish a review rubric, dual-review a subset of answers, track accuracy by topic area, and gate releases when the judged correctness rate falls below the board’s threshold.
When leadership names the reviewer-marked share as the success target, treat that rate as a release gate rather than a vanity dashboard: freeze the prompt and retrieval sources when the rate dips, then retest on the same sample before promoting changes. That keeps business-facing accuracy operational instead of ceremonial.