A municipal fish-hatchery summary shares many words with the brochure but invents a fish count. The overlap score looks fine; a faithfulness check fails. What should the team conclude?
Select an answer to reveal the explanation.
Short Explanation
Think of a hatchery summary that steals brochure words and still invents a fish count. Overlap can look fine while faithfulness fails. Shared words are not proof the facts were grounded.
Full Explanation
A hatchery summary that shares many brochure words yet invents a fish count shows why overlap is not proof the facts were grounded. Lexical overlap rewards shared phrasing; faithfulness asks whether each claim is supported by source evidence. The municipal team should conclude that a fine overlap score can coexist with a failed faithfulness check whenever invented numbers ride along with borrowed wording from the brochure or press kit.
Treating high overlap as automatic faithfulness collapses two different evaluation questions into one misleading green light. Faithfulness and overlap are not the same metric, and conflating them hides numeric hallucinations in public content. Accepting invented counts because enough brochure words appear would normalize unsupported claims in hatchery reports that residents and partners may trust as official.
Exam caveat: on AIF-style items, “sounds supported because words match” is a common trap. Always separate surface similarity from groundedness when the stem mentions invented quantities, serials, or other concrete civic facts that must appear in the source.
Operational check: for hatchery or other numeric municipal summaries, run overlap for wording quality and a separate faithfulness pass that flags unsupported counts; require a human sign-off whenever a numeric claim is not present in the brochure or retrieved source chunk before publication.