A deployed model in SAS Viya shows concept drift — the relationship between features and the target has changed but the input distributions remain stable. Which monitoring approach best detects concept drift (as opposed to data drift)?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you need to watch outcome-based performance metrics like KS, Gini, and AUC on new labeled data because concept drift shows up as degraded predictive accuracy even when inputs look stable.
Full Explanation
The correct approach is monitoring model performance metrics (KS, Gini, AUC) on new labeled data over time to detect degradation in predictive accuracy. Concept drift manifests as degradation in predictive accuracy — the model's predictions become less aligned with actual outcomes even though input distributions appear stable. Monitoring outcome-based metrics on labeled data (where outcomes are eventually known) directly measures concept drift. Monitoring PSI on input variables to detect distribution changes monitors data drift, not concept drift. Monitoring missing-value counts per variable across scoring batches monitors data quality issues, not predictive relationship changes. Monitoring model API calls per day monitors operational traffic, not model accuracy.