SAS Model Manager's monitoring framework supports model performance monitoring through scheduled monitoring jobs. What type of data is required as input to a performance monitoring job that computes outcome-based metrics (KS, Gini, AUC)?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal — you need a matched dataset that joins predicted probabilities from the scoring period with actual observed outcomes for those same records. Outcome-based metrics like KS, Gini, and AUC can't be computed from predictions alone.
Full Explanation
The correct input is a matched dataset that joins predicted probabilities from the scoring period with actual observed outcomes for those same records. Outcome-based metrics such as KS, Gini, AUC, and Brier score require both predicted scores and actual outcomes for the same records. This 'actuals matching' is the key challenge in production monitoring, since outcomes like loan default may take months to materialize. The production scoring dataset with predicted probabilities and input features only, but no target variable, supports distribution-based monitoring such as PSI, not outcome-based metrics. The baseline distribution dataset from training is the baseline for PSI computation. The raw model ASTORE file is the model artifact, not a monitoring input dataset.