Meridian's customer-service virtual assistant continues receiving the same mix of rebooking, baggage, and loyalty questions it was trained on, but its accuracy at correctly escalating complex cases to a human agent has quietly declined over a year. No input patterns have changed. What should the project manager suspect is happening?
Select an answer to reveal the explanation.
Short Explanation
Same questions coming in, worse answers going out — that's model drift. The world's relationship between 'what a customer asks' and 'what should happen next' has quietly shifted, and the model's stale logic hasn't kept pace.
Full Explanation
Model drift refers to a decline in a model's predictive relationship even when the input data distribution itself is stable — the model's learned mapping between inputs and correct outputs has gone stale relative to how the real world now behaves. Here, the mix of questions Meridian's chatbot receives hasn't changed, but what counts as a case needing human escalation may have shifted (new policies, new edge cases in rebooking rules, evolving customer expectations), so the model's older learned patterns no longer produce the right escalation decisions. CPMAI's Model Evaluation and Maintenance task treats data drift and model drift as related but distinct failure modes: data drift is a shift in the inputs, model drift is decay in the input-output relationship. The stem explicitly rules out data drift by stating input patterns are unchanged. A labeling mistake present from the start would show up as poor accuracy from day one, not a gradual decline after a year of good performance. A failed Data Understanding phase would have surfaced during the original project's Go/No-Go checks, not a year into stable production operation. The exam distinction to hold onto: watch the inputs for data drift, watch the input-output relationship for model drift, and both require ongoing monitoring after deployment, not just a one-time evaluation before launch.