After a permits intake form redesign, feature distributions shift and the live model’s inputs no longer resemble training. What should monitoring detect?

Detecting data drift in application feature distributions after the intake change is correct because a permits form redesign alters what applicants enter and how fields are encoded, so live feature distributions diverge from the training baseline. Production monitoring that compares current feature stats to training reference distributions is how municipal teams notice that the live model’s inputs no longer resemble what it learned. Detecting that drift is the prerequisite to deciding whether to retrain, remap features, or fix upstream form logic.

Monitoring concept drift in shifted feature distributions after the intake change fails because the form redesign changed the input distributions themselves, not the link between those inputs and approval outcomes; concept drift would require outcome labels and describes a different failure mode. Checking for data quality faults in shifted feature distributions after the intake change fails because the issue is a wholesale shift in feature distributions after the redesign, not isolated missing or malformed records; quality checks won’t show that live inputs diverge from the training baseline. Tracking model performance decline over shifted features after the intake change fails because by the time approval labels arrive the drift has already affected scoring; the question asks about detecting that inputs no longer resemble training, which can be done immediately from live feature stats without waiting for outcomes.

Exam caveat: intake, form, or upstream application changes paired with “inputs no longer resemble training” point to data drift monitoring—not concept drift in shifted feature distributions after the intake change, data quality faults in shifted feature distributions after the intake change, or model performance decline over shifted features after the intake change.

Operational check: enable or review drift monitors on the permits model’s key features against the training baseline, simulate or observe post-redesign traffic, confirm drift alerts fire when distributions shift, and route the alert to analysis before any blind retrain.