A data science team trains a classification model on a labeled dataset and achieves 99% accuracy when they evaluate it on the exact same rows used for training. When they run the same model against a separate set of labeled rows that were held out and never used during training, accuracy drops to 61%. What does this gap between the two accuracy figures most likely indicate?
Select an answer to reveal the explanation.
Short Explanation
A model that scores 99% on the very rows it studied but only 61% on rows it has never seen has basically memorized the training set instead of learning the general patterns underneath it. Picture a student who memorizes the exact practice questions word for word but cannot answer a rephrased version of the same question, that is the same failure showing up here. The huge gap between the two scores is the giveaway. The wrong turns to avoid: calling this underfitting would mean the model performs poorly everywhere, including on the training data itself, which is not what happened since it aced that set. Blaming the held-out rows for being mislabeled is an assumption nothing in the scenario supports. And reaching for a completely different training approach built around trial-and-error rewards does not address why a model that already learned this way is failing to generalize.
Full Explanation
The correct answer is A. A model that scores 99% on the exact rows it trained on but only 61% on data it has never seen has memorized specifics of the training set rather than learning patterns that generalize, which is the classic signature of overfitting. Option B is incorrect because underfitting would produce poor performance on the training data as well as on new data, and this model performed extremely well on the training rows, ruling out underfitting. Option C is incorrect because assuming the held-out data is mislabeled is not supported by anything in the scenario; a large accuracy gap between training and held-out performance is the expected symptom of overfitting, not evidence of a labeling error. Option D is incorrect because switching to a reward-based training approach does not address the underlying cause of the gap, which is that the current model captured noise and specifics of the training rows instead of general patterns, a problem tied to model complexity or training data rather than the category of learning algorithm. The large train-versus-holdout gap is the key diagnostic detail pointing to overfitting.