A public-health team fine-tunes a vaccination-FAQ model and must catch quality regressions across checkpoints. What should they do after each training checkpoint?
Select an answer to reveal the explanation.
Short Explanation
Each checkpoint is a new batch of vaccine answers—taste-test it on the same FAQ eval suite before it goes live. Monitoring for regression keeps a bad checkpoint from replacing a good helper. Running a fresh public QA sample after each checkpoint tracks general shifts, but you still won't catch vaccination-FAQ regressions without your fixed eval suite.
Full Explanation
Running the fixed vaccination-FAQ evaluation suite and monitoring for regression before promotion is what a public-health team should do after each fine-tuning checkpoint. Checkpoint-level evaluation catches quality drops early, keeps a comparable scorecard across training, and prevents unsafe or inaccurate FAQ answers from reaching residents. Fixed suites make regressions detectable rather than anecdotal. Running the fixed vaccination-FAQ evaluation suite and promoting the lowest training loss value still bases promotion on model fit, not vaccination-FAQ correctness; training loss can fall while answers become less accurate, unsafe, or ungrounded, so promoting on loss ignores the fixed FAQ evaluation gate. Running the fixed vaccination-FAQ evaluation suite after final training and recording regression for audit comes too late to catch quality regressions across checkpoints; without per-checkpoint evaluation to block promotion, regressions can accumulate undetected and final-only recording does not prevent unsafe FAQ answers from advancing. Running a fresh public QA sample after each checkpoint and monitoring for regression before promotion tracks general capability shifts, but it is not the fixed vaccination-FAQ suite and will not catch domain-specific accuracy, groundedness, or safety regressions across checkpoints. Exam caveat: regression monitoring belongs in the fine-tune loop, not only at final cutover. When stems mention catching quality regressions across checkpoints, answer with fixed eval suites after each checkpoint—not running the suite yet promoting the lowest training loss value. Operational check: after each checkpoint, run the vaccination-FAQ suite for accuracy, groundedness, and safety flags; block promotion on any metric below the baseline threshold; retain scored checkpoint artifacts for audit and rollback.
Prefer to study offline? Printable PDF study book + test engine ($7.50, unofficial). These questions stay free here.