A city establishes a performance baseline for its permit-approval-time prediction model by recording its accuracy on a representative validation set at launch. Several months later, monitoring shows live predictions have drifted noticeably from that recorded baseline. What does this drift most directly indicate the team should do next?
Select an answer to reveal the explanation.
Short Explanation
Think of the baseline as a doctor's note of your normal blood pressure — if a later reading is way off, the useful next step is figuring out why, not throwing out the original reading or panicking and changing medication blind. A performance baseline exists precisely so drift is detectable; once it's detected, the right move is investigating what in the data or environment actually changed. That diagnosis should come before deciding on a fix.
Full Explanation
A performance baseline exists to give the team a reference point for detecting when a model's live behavior has diverged from its expected behavior, and drift against that baseline is a signal that something about the underlying data distribution, feature relationships, or real-world conditions has likely changed since launch, such as new permitting rules or shifting application volumes. The correct next step is investigating the cause of the drift, since understanding whether it stems from data quality issues, a genuine distribution shift, or an external process change determines what fix, if any, is appropriate. Discarding the baseline treats the tool that caught the problem as disposable, when in fact the baseline is what made the drift visible in the first place and remains the reference point for judging whether a fix worked. Concluding the original measurement was wrong ignores that drift is an expected phenomenon in deployed ML systems as real-world conditions change over time; a correctly calculated baseline can still be legitimately drifted from later. Immediately retraining with default settings and no investigation risks masking the actual cause, for example retraining on data that still reflects a data-quality bug, and skips the diagnostic step that determines whether retraining is even the right fix versus a pipeline correction. Scope note: not all drift requires action; small, expected seasonal variation may fall within acceptable tolerance. Operational check: compare current feature distributions against the baseline's training-time distributions to localize where the drift is occurring before deciding on a remediation.