A public health department evaluates generated symptom-report summaries using BERTScore-based semantic similarity instead of exact word-overlap metrics. A reviewer asks why the team moved away from lexical-overlap scoring for this task. What is the most accurate justification?
Select an answer to reveal the explanation.
Short Explanation
Think of two nurses describing the same patient's symptoms in their own words — a lexical checker would flag them as different, but anyone who actually reads both notes can tell they mean the same thing. BERTScore uses contextual embeddings to judge meaning rather than exact word matches, so it can credit a paraphrased summary that says the right thing differently. That's the whole reason it fits paraphrase-tolerant evaluation better than ROUGE here.
Full Explanation
BERTScore evaluates generated text by comparing contextual embeddings of the candidate and reference text rather than counting shared word sequences, which lets it recognize that two differently worded summaries convey the same clinical meaning and score them both appropriately. This makes it a better fit than lexical-overlap metrics like BLEU or ROUGE specifically when paraphrasing is expected and meaning preservation matters more than exact wording, as in symptom-report summarization where clinicians may phrase the same finding several valid ways. The option claiming ROUGE cannot be computed on medical text is false; ROUGE is vocabulary-agnostic and can be applied to any text, including specialized domains, it simply measures the wrong thing for paraphrase tolerance rather than being technically inapplicable. The option claiming BERTScore eliminates the need for reference summaries misdescribes the metric; BERTScore is still a reference-based comparison, it just compares meaning through embeddings instead of exact words, rather than judging fluency in isolation. The option framing the switch as primarily about computation speed misidentifies the actual justification; BERTScore is generally more computationally expensive than simple n-gram counting because it requires running a language model to produce embeddings, so speed was not the driver, semantic fidelity was. Scope note: embedding-based metrics still depend on the quality of the underlying embedding model and can miss domain-specific nuance it wasn't trained on. Operational check: manually review a sample of summaries with mid-range BERTScore results to confirm the score aligns with a clinician's judgment of accuracy.