SAS Model Manager's monitoring framework supports model performance monitoring through scheduled monitoring jobs. What type of data is required as input to a performance monitoring job that computes outcome-based metrics (KS, Gini, AUC)?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal — b is correct because outcome-based monitoring metrics (KS, Gini, AUC, Brier score) require both the model's predicted scores and the actual observed outcomes for the same records. This 'actuals matching' is the key challenge in production monitoring, since outcomes like loan default may take months to materialize.
Full explanation below image
Full Explanation
B is correct because outcome-based monitoring metrics (KS, Gini, AUC, Brier score) require both the model's predicted scores and the actual observed outcomes for the same records. This 'actuals matching' is the key challenge in production monitoring, since outcomes like loan default may take months to materialize. A (predictions only) enables distribution-based monitoring (PSI) but not outcome-based metrics. C is the baseline for PSI computation. D is wrong; the ASTORE is the model artifact, not a monitoring input dataset.