The laundry forecast keeps drifting for six weeks after the new dorm opens. A technician offers to hotfix the dryer card-reader firmware on Friday night and skip a new evaluation. Which MLOps response is appropriate?
Select an answer to reveal the explanation.
Short Explanation
Think of rewriting the playbook after the new dorm changes Sunday, not taping the card reader on Friday night. Retrain on recent seasons, evaluate again, then release. Silencing alerts or swapping the metric just hides the drift.
Full Explanation
Six weeks of sustained drift after the dorm opened means the laundry forecast no longer matches current campus behavior. The sound MLOps response is a controlled retrain on recent seasons that include the new occupancy pattern, a fresh evaluation on held-out recent data, and only then a planned release. That sequence keeps change management intact: you rebuild on representative history, prove quality, and promote deliberately instead of improvising in production.
An ad-hoc firmware hotfix on the dryer card reader with no new evaluation confuses infrastructure tinkering with model remediation. Even if the reader has bugs, skipping evaluation after a production-impacting change leaves the team blind to whether forecasts improved, worsened, or merely looked quieter for a night.
Deleting monitoring alerts so drift looks resolved is alert suppression, not remediation. The underlying mismatch between live Sunday demand and the old model remains; you only removed the signal that would have forced action.
Switching the metric to RMSE and declaring the old model fine is metric shopping. Changing the scorecard to make an outdated model appear acceptable does not restore accuracy on the new dorm's usage pattern and undermines trust in evaluation.
Exam caveat: retrain-without-evaluate is a common trap. Fresh weights on recent data still need a comparable test protocol before cutover, especially when campus population shifted.
Operational check: freeze a recent evaluation slice that includes post-dorm Sundays, train only on allowed history that does not leak that slice, compare champion versus challenger on the same metrics and latency budget, then promote through the normal release path with monitoring left on.