A permitting model is silently retrained weekly with no change tickets. What should the auditor recommend?
Select an answer to reveal the explanation.
Short Explanation
Don't mistake a registry record with rollback versions for change control; you still need tickets and approval before a weekly retrain touches permits. Put retraining under change control with tickets and review. Don't confuse traceability logs or data-science peer reviews with tickets; you still need governed change control before quiet success reaches permit decisions.
Full Explanation
Subjecting permitting-model retraining to change control with tickets, review, and approval proportionate to municipal impact works because retraining is a material change to decision logic even when accuracy looks stable. Weekly silent retrains alter weights, feature influence, and edge-case behavior that applicants experience as shifting permit outcomes. Change tickets create accountability, testing evidence, rollback points, and communication paths for operations and oversight. The correct recommendation treats retraining as governed change, not invisible background maintenance.
Recording weekly retrains in the model registry with metrics and retaining prior versions fails because retrospective logging and rollback points do not replace pre-promotion tickets, review, and approval appropriate to municipal impact. Logging weekly datasets, hyperparameters, and validation scores in experiment tracking fails conceptually: traceability is not authorization; successful changes are still changes that require a change ticket and review before they affect permitting. Using peer review within data science to approve weekly retrains and release validated models fails because internal technical sign-off is not governed change: it lacks municipal impact review, operations and oversight involvement, and a formal audit trail for disputes.
Exam caveat: highly automated retraining can still be change-controlled via pre-approved pipelines with automatic evidence capture—control does not always mean slow manual CAB for every epoch, but it does mean authorized, recorded, reversible process. Operational check: require a change record for each retrain promotion, attach evaluation results and data-period used, define rollback to the prior registry version, and sample weekly jobs to confirm none promoted outside the change process.