A team trains a deep neural network that achieves 99% accuracy on the training set, but only 61% accuracy on new, unseen data from the same problem domain. Which phenomenon best explains this gap?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of a student who memorizes the exact answers to last year's practice exam instead of actually learning the subject — they'll crush that specific test, then bomb the real exam with new questions. That's exactly what's happening here: the model has memorized quirks of the training set rather than learning patterns that generalize, and that's overfitting, answer D. Underfitting is the opposite problem — it means the model performs poorly on both training and new data, which isn't the case here since training accuracy is sky-high. Vanishing gradients would actually show up as the model struggling to learn well anywhere, including on the training set. And label leakage is a data problem where the answer sneaks into the features — it usually produces suspiciously perfect performance everywhere, not this kind of clean train/test split.
Full explanation below image
Full Explanation
The described scenario — very high training accuracy paired with much lower accuracy on unseen data from the same domain — is the textbook signature of overfitting. Overfitting occurs when a model has enough capacity (or is trained long enough) that it starts capturing noise, idiosyncrasies, and sample-specific details of the training set rather than the true underlying patterns that generalize to new data. The large gap between training and validation/test performance is often called the "generalization gap," and it is the primary indicator practitioners use to diagnose overfitting.
The first distractor, underfitting, describes the opposite failure mode: a model too simple or undertrained to capture even the patterns in the training data, which shows up as poor performance on both the training set and unseen data. Since training accuracy here is 99%, underfitting is ruled out.
The second distractor, vanishing gradients, is a training-dynamics problem where gradients shrink as they propagate backward through many layers, causing early layers to learn very slowly or not at all. This typically manifests as the model failing to reach high accuracy even on the training set, which contradicts the 99% figure given.
The third distractor, label leakage, happens when a feature in the training data inadvertently encodes the target directly (e.g., a proxy variable), causing artificially inflated performance. While leakage can cause a similar-looking gap if the leaking feature is unavailable or different at inference time, it is a data-pipeline problem, not a description of what happens inside the model as it learns, and is not the most direct or standard explanation for this classic pattern.
Memory aid: "high train, low test = overfit; low train, low test = underfit." Common remedies for overfitting include regularization (L1/L2), dropout, early stopping, data augmentation, and gathering more training data.