Domain 5: Model Deployment & Monitoring

SAS AI and Machine Learning · 49 questions

• In SAS Model Manager, what is the purpose of designating a model as the 'champion' model in a champion/challenger deployment pattern?
• A data scientist uses PROC ASSESS in SAS to evaluate a deployed logistic regression model. Which capability does PROC ASSESS provide that directly addresses model drift detection?
• In SAS Model Manager, a data scientist registers a SAS score code file. Which file extension does SAS Model Manager require for SAS score code?
• A MLOps engineer configures model monitoring in SAS Model Manager to detect data drift. Which statistical measure is commonly used to compare the distribution of an input variable between training data and current production data?
• A data scientist wants to retrieve a specific version of a registered model from SAS Model Manager via the SAS Viya REST API. Which resource path is correct?
• During model monitoring in SAS, a data scientist observes that the Gini coefficient for a credit risk model has declined from 0.62 at deployment to 0.41 six months later. What does this indicate?
• A model built in SAS Model Studio has been registered in SAS Model Manager. A data engineer wants to generate batch scores for 10 million new records overnight. Which SAS component is most appropriate for orchestrating this batch scoring job at scale?
• In SAS Model Manager, which term describes the automated process of detecting when a model's predictive performance has degraded enough to trigger an alert or retraining workflow?
• A data scientist needs to load a Python-trained scikit-learn model into SAS Model Manager for deployment alongside SAS models. Which SAS capability enables this?
• In SAS Model Manager, a data scientist wants to publish a model to a SAS Micro Analytic Service (MAS) destination for real-time scoring. Which statement best describes MAS?
• A model deployed in SAS Model Manager is monitored using PROC ASSESS. The data scientist wants to track Kolmogorov-Smirnov (KS) statistic over monthly scoring runs. What does a significantly increasing KS statistic over time indicate?
• In a SAS Viya environment, a data scientist wants to automate monthly model retraining when data drift is detected. Which SAS component is most appropriate for orchestrating this automated workflow?
• A model registry in SAS Model Manager contains version 1 (champion) and version 2 (challenger) of a credit risk model. Version 2 shows a Gini coefficient of 0.71 vs. 0.64 for version 1 in the challenger test. What action should the data scientist recommend?
• In SAS Viya's Model Manager, what is the purpose of the 'model comparison report' generated during champion/challenger evaluation?
• In SAS Model Studio, a data scientist selects the 'Gradient Boosting' node and wants to tune the number of trees using AutoTune. Which SAS Viya feature enables automatic hyperparameter search for gradient boosting?
• In SAS Model Manager, when a data scientist publishes a model to the 'SAS Micro Analytic Service' destination, which score code format is required?
• A model monitoring system in SAS detects that input variable 'creditutilization' has a PSI (Population Stability Index) of 0.31 over the past month. How should the data scientist interpret this?
• A data science team uses SAS Model Manager to track multiple model versions. Which version control capability does SAS Model Manager provide?
• A deployed model in SAS Viya shows concept drift — the relationship between features and the target has changed but the input distributions remain stable. Which monitoring approach best detects concept drift (as opposed to data drift)?
• In SAS Model Manager, a data scientist configures a performance monitoring job that compares model predictions to actual outcomes. Which SAS procedure is most commonly used within this monitoring workflow to generate model fit statistics?
• A data scientist registers a SAS model in the Model Manager repository and must specify the model's input and output variable definitions. Which input variable property must be correctly specified to ensure downstream scoring pipelines work correctly?
• A data scientist uses SAS Model Studio to build an automated machine learning (AutoML) pipeline. After model comparison, the best model is a gradient boosting model. Which SAS Model Studio action makes the selected model available for production scoring in SAS Model Manager?
• In SAS Model Manager, a data scientist runs a monitoring report and observes that the model's KS statistic dropped from 0.55 (baseline) to 0.32 (current month). The input variable PSI values are all below 0.10. What type of drift does this pattern most likely indicate?
• A SAS Model Manager project has a model registered with only SAS score code (no ASTORE). How can this model be deployed for scoring?
• Which SAS Viya service provides REST API endpoints for submitting jobs, monitoring status, and retrieving results for integration with external orchestration tools?
• A monitoring threshold triggers PSI > 0.20 as an alert. What type of drift does PSI specifically measure?
• In SAS Model Manager, what does the formal 'model validation' workflow step accomplish before deployment?
• PROC ASSESS with the FITPLOT option is run after logistic regression. What does the FITPLOT produce?
• A model governance team wants to ensure a model does not produce significantly different false positive rates across demographic groups before deployment. Where in SAS Model Manager governance is this addressed?
• When a champion model is replaced by a challenger in SAS Model Manager, what happens to the previous champion?
• A data scientist registers a model in SAS Model Manager and wants to test it using SAS Micro Analytic Service (MAS) before champion promotion. What endpoint pattern does MAS expose for real-time score code testing?
• In SAS Model Manager, the champion/challenger framework is configured for a credit scoring model. The challenger receives 20% of scoring requests. After 30 days, the challenger's Gini (0.71) exceeds the champion's Gini (0.66) with statistical significance. What action does the champion/challenger framework enable?
• In SAS Model Manager, a model monitoring job runs weekly and computes PSI and KS statistics. The monitoring job configuration requires specifying a 'baseline dataset.' What role does the baseline dataset play?
• A production model's KS statistic drops from 0.65 to 0.41 over 6 months. PSI for input features remains below 0.10. What is the most likely explanation?
• A team deploys a new fraud model using canary deployment with 15% of traffic. The champion continues receiving 85%. After 2 weeks, the canary achieves statistically significantly better precision. What distinguishes canary deployment from A/B testing in this context?
• PROC ASTORE is used to score new observations in SAS. Which PROC ASTORE statement option specifies where the model's input variables and types are defined?
• A blue-green deployment is set up for a SAS Micro Analytic Service scoring endpoint. The 'green' environment contains the new model version. What is the final atomic step that completes the deployment?
• In shadow deployment for a new ML model, what distinguishes it from both A/B testing and canary deployment?
• SAS Model Manager tracks model versions. A data scientist wants to programmatically list all versions of a registered model via the REST API. Which endpoint pattern is correct?
• Model risk management standards (SR 11-7) require that models used in risk decisions be validated by an independent party. In SAS Model Manager, which workflow element supports the independent validation requirement?
• SAS Model Manager's monitoring framework supports model performance monitoring through scheduled monitoring jobs. What type of data is required as input to a performance monitoring job that computes outcome-based metrics (KS, Gini, AUC)?
• A model governance team uses SAS Model Manager to enforce a policy that all models must be validated before promotion to production. Which SAS Model Manager feature enforces this workflow policy?
• A data scientist registers a Python XGBoost model in SAS Model Manager by uploading a ZIP file. After registration, she wants to run the model's score code on a test dataset inside SAS Model Manager. What must be true for the scoring to succeed?
• PROC ASSESS is run with the FITPLOT option on a logistic regression model. What does the FITPLOT calibration plot display?
• A deployed scoring model accumulates 30 days of production predictions. PSI = 0.22 is detected for the predicted score distribution. According to standard PSI thresholds, what action should be taken?
• A model risk management team adds a fairness assessment requirement: models must demonstrate that precision does not differ by more than 5 percentage points across demographic groups. Which SAS Model Manager or SAS Viya workflow step supports this group-stratified fairness metric computation?
• PROC ASTORE DESCRIBE is run on a deployed model. What information is returned in the PROC ASTORE DESCRIBE output?
• In SAS Viya, the SAS Event Stream Processing (ESP) engine is used to score a fraud detection model on streaming transaction data. Which model deployment format does ESP natively support for real-time scoring?
• The PMML format is used to export a logistic regression model from SAS for deployment in a non-SAS scoring environment. Which SAS procedure can export a fitted logistic regression model to PMML?