A model scores nearly perfectly on the training set but performs poorly on held-out validation and test examples. Which phenomenon best describes this behavior?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Okay, let's dive in. Imagine your boss walks in after a great training run: "Accuracy is 99.9%—ship it!" You peek at the test set and… yikes. The model basically memorized the homework answer key and flunked the pop quiz. That's overfitting. Think of it like a student who crams every example problem word-for-word but freezes when the numbers change. Exam trap: people mix this up with underfitting (bad on train and test) or "didn't converge" (train loss never drops). If train looks great and new data looks bad, you've overfit. Regularize, get more data, simplify, early-stop—don't celebrate train scores alone. You've got this.
Full explanation below image
Full Explanation
Overfitting occurs when a learning algorithm fits the training sample too closely—including noise, outliers, and spurious correlations—so that predictive performance degrades on new draws from the same population. Empirically it appears as a large gap between training metrics and validation or test metrics: very low train error paired with substantially higher error on held-out data. High-capacity models (deep nets, high-degree polynomials, unpruned trees) and small or noisy datasets make this gap more likely.
Underfitting is the complementary failure mode: the hypothesis class or training regime is insufficient to capture the signal, so error remains high on both training and test sets. Calling the described pattern "healthy generalization" is incorrect because generalization is judged on unseen data; excellent train performance alone is insufficient evidence of readiness for deployment. Optimization failure would present as an inability to drive training loss down; here the model has already achieved near-perfect train performance, so the optimizer has effectively fit the training objective.
Mitigations include collecting more diverse labeled data, simplifying capacity, regularization (L1/L2, dropout, weight decay), data augmentation, early stopping on a validation set, and ensemble methods that reduce variance. Best practice is to monitor train-versus-validation curves throughout training and to reserve a true test set for a final, one-time estimate of generalization. Underlying principle: generalization fails when the model fits sample-specific noise rather than population signal. Best practice watches the train–validation gap early, applies capacity control before celebrating train accuracy, and never uses the final test set for iterative tuning. Memory aid: memorizes the homework and fails the exam equals overfitting; fails the homework too equals underfitting.