A housing-authority waitlist analytics team notices feature distributions shift after a new e-permitting form launches. What should production monitoring detect?
Select an answer to reveal the explanation.
Short Explanation
You’re right to watch for input feature drift when the new form changes how fields are filled. Compare live feature distributions with the training baseline so you can catch shifts that may affect waitlist scores.
Full Explanation
Production monitoring should detect input feature drift by comparing live field distributions with the training baseline. A new e-permitting form can change how applicants fill out fields, shifting the inputs used to score the waitlist; scores may then degrade silently even if the code still runs. Monitoring key features and related data quality helps reveal this risk. Checking whether the office printer has paper does not measure model inputs, and checking whether the waitlist portal’s visual theme stays consistent measures appearance rather than feature distributions. Checking DNS latency to a social media site is also unrelated to shifts in housing-application fields. Exam caveat: the signal to monitor here is drift in model inputs caused by changed field distributions, not an unrelated operational condition. Practical check: after the form launches, compare live feature histograms or PSI/KS-style drift scores with the training baseline, alert when thresholds are exceeded, and review the changes before relying on waitlist decisions made from drifted inputs.