A public-health eval harness flags rising hallucination rates after a prompt edit. What recovery action fits?
Select an answer to reveal the explanation.
Short Explanation
Eval says hallucinations spiked after the prompt change—roll that edit back before the clinic bot teaches fiction. Don't push it to staged UAT, don't relax the eval thresholds to hide the spike, and don't crank up the temperature to paper over the problem.
Full Explanation
When a public-health eval harness flags rising hallucination rates after a prompt edit, the correct recovery is to roll back that prompt change based on the evaluation signal. Eval harnesses exist to catch regression before clinic bots mislead residents; rising hallucination metrics after a specific edit establish a causal change-control case for revert.
Promoting the edited prompt for staged user acceptance testing fails because rising hallucination rates are a regression signal; moving it toward users before the metric clears baseline risks exposing residents to faulty outputs. Tuning the eval harness thresholds to accommodate higher hallucination rates fails conceptually: relabeling unacceptable risk as acceptable does not reduce factual error risk to patients or the public. Raising generation temperature to increase response diversity in evaluation runs fails because it masks the underlying prompt regression rather than fixing it, and variability is not a substitute for factual accuracy.
Exam caveat: confirm the metric spike is attributable to the prompt edit (same fixtures, same model) rather than a broken scorer or data-label change before you finalize the rollback narrative. Operational check: revert the prompt to the last green revision, re-run the hallucination suite, require scores at or below the prior baseline, then gate further prompt merges on the same harness.