A photo-classification startup trains a model on its labeled training set and it reaches 99 percent accuracy there, but the same model correctly labels only 61 percent of brand-new photos customers submit once it goes live. The live photos come from the same product catalog and were labeled using the same rules as the training set. Which concept best explains the gap between training and live performance?
Select an answer to reveal the explanation.
Short Explanation
This is the classic case of a model that crammed for the test instead of actually learning the material. Ninety-nine percent on the training photos sounds great until you realize the model may have just memorized the exact pixels and quirks of those specific images rather than picking up on the broader patterns that make a product recognizable. So when a fresh photo comes in, one it has never seen, it stumbles. The tempting wrong answers all point somewhere else: one suggests the model never learned enough in the first place, but that would mean it struggled on the training data too, and here it aced that part. Another blames a shift in where the new photos are coming from, but the setup tells us the catalog and labeling process stayed the same, so nothing actually changed on that front. A third blames uneven categories in the training set, which usually shows up as trouble with specific underrepresented categories, not a blanket drop across the board. The real story here is a model that fit the training set too tightly and lost the ability to generalize.
Full Explanation
The correct answer is A. A model that scores 99 percent on the data it was trained on but drops sharply on new, similarly-sourced data has memorized quirks of the training photos rather than learning patterns that generalize, which is the definition of overfitting. Option B is incorrect because underfitting produces low accuracy on the training data itself, not the 99 percent score described here; an underfit model has not learned enough to do well anywhere. Option C is incorrect because data drift refers to the incoming data's characteristics changing over time or across sources, but the stem states the live photos come from the same catalog and are labeled the same way, so there is no evidence the source shifted. Option D is incorrect because class imbalance describes an uneven number of examples per category in the training set, which typically shows up as poor performance on the rare categories specifically, not as a broad gap between training and live accuracy across the board.