A media streaming team wants a new homepage recommendation model live tonight. The offline suite that scores ranking quality on last month's holdout has not been run. What should happen before the model is promoted?
Select an answer to reveal the explanation.
Short Explanation
You would not ship a parcel that skipped the scale at the dock. Hold the homepage promotion until the offline ranking checks pass on the agreed holdout. The trap is treating tonight's live traffic as a substitute for that gate.
Full Explanation
Promotion is the step that puts a model on the live path, and the offline suite is the gate in front of that step. The ranking checks on last month's holdout have not been run, so the model has not earned promotion yet. Homepage viewers are the people the gate is there to protect, which is why the suite has to run before they see a new ranking. Holding the release until those checks pass is the required order: offline validation, then promotion. Promoting tonight and calling complaints the test moves the gate to after viewers are already affected, which is monitoring, not a pre-promotion check. Training loss on the rows the model just fit is not the holdout suite; a low training loss can coexist with a failed ranking check. Copying files into a folder and labeling them production without a pass does not run the suite, and leaving the homepage unchanged means nothing was validated or promoted. Exam caveat: an online watch after release does not replace the offline gate that has to pass first. Operational check: confirm the promotion job reads a passed offline ranking report and refuses to mark the model live when that report is missing.