A manufacturer validated its weld-defect model only on sharp studio photos. Plants submit glare, dust, and cropped beads, and none of those photos are in the validation set. What should the architect require before promotion?
Select an answer to reveal the explanation.
Short Explanation
A road test on an empty lot says little about driving in the rain. You add the glare, dust, and cropped photos the plants actually send to the validation set before this weld model is promoted. A longer prompt does not create the missing cases.
Full Explanation
A custom model is ready to promote only when validation has seen the inputs production will send. This weld model was scored on sharp studio photos, while the plants send glare, dust, and cropped beads, so those dirty-photo cases belong in the validation set before promotion. Promoting on the studio photos alone and treating those sharp images as a stand-in for plant shots leaves the real failure mode unmeasured. Dropping the validation set and lengthening the prompt so the model is told to see through dust changes instructions and still does not score a dirty photo. Scoring the model only on the clear studio set and leaving every plant photo out of validation repeats the same gap and rewards a match to the easy images. The validation set is the evidence for the release, and evidence that omits the plant photos cannot support it. Exam caveat: photos added for this check stay out of training, or the promotion result is no longer a holdout. Operational check: score a labeled batch of real plant photos, confirm those files are named in the validation report, and hold promotion while the dirty cases are absent.