A historic lightship education desk has two docents independently score a week of generated tour lines for unfair group stereotypes and invented dates. Which monitoring method is that?
Select an answer to reveal the explanation.
Short Explanation
Think of two docents independently scoring a week of tour lines for stereotypes and invented dates. That is a human audit. ROUGE and an endpoint size are not that review.
Full Explanation
Two historic lightship docents independently scoring a week of generated tour lines for unfair stereotypes and invented dates is a human audit of outputs for bias, trustworthiness, and truthfulness. Human evaluation catches harms automatic n-gram metrics miss: stereotyping, fabricated maritime dates, and tone issues that still score well on overlap measures. Responsible monitoring mixes automated checks with structured human review, especially for public education content.
ROUGE only fails because n-gram overlap with references does not detect stereotypes or invented dates when those flaws remain fluent. Provisioned throughput fails because endpoint capacity sizing is an infrastructure concern, not an audit method for bias or truthfulness. From-scratch pre-training fails because running a weekend crawl does not constitute two docents reviewing live tour lines for harm.
Exam caveat: human audits need rubrics, independent scorers, and disagreement resolution—informal hallway opinions are weaker evidence than a documented review protocol. Operational check: schedule recurring dual-docent reviews on a fixed sample of tour lines, log stereotype and factuality scores with examples, gate content release on audit thresholds, and feed failing patterns back into prompts, filters, or data fixes.