After SFT, resident satisfaction with a city assistant has stalled despite more gold demos. What should the team consider next?
Select an answer to reveal the explanation.
Short Explanation
If extra gold demos stopped moving the resident-satisfaction needle, more SFT is probably pouring water in a full bucket. That’s when preference optimization like DPO deserves a shot.
Full Explanation
After SFT, resident satisfaction with a city assistant can plateau even as teams add more gold demos, because additional supervised clones mostly reinforce the same behaviors without teaching relative preferences among already-plausible replies. Moving to preference optimization such as DPO when more SFT no longer lifts civic satisfaction targets pairwise (or ranked) human judgments that distinguish better municipal answers from merely acceptable ones. That stage shift is the usual remedy for an SFT plateau. Keeping identical SFT clones forever without measuring preference gaps burns annotation budget without addressing the satisfaction ceiling. Deleting all supervised data and training only with unfiltered web scrapes destroys civic tone and introduces uncontrolled content. Disabling evaluation and shipping whichever checkpoint finished training last abandons quality gates residents rely on. On the exam, when municipal SFT gains stall, prefer DPO-style preference optimization over endless identical demos; distractors clone SFT forever, switch to scrapes, or ship without eval. Caveat: preference data quality matters—if raters disagree on “good” city answers, DPO amplifies noise; calibrate rubrics first. Operational check: confirm satisfaction metrics flat across recent SFT runs, collect preferred/rejected resident-assistant pairs, train DPO, and compare satisfaction / win-rate against the best SFT checkpoint before cutover.