During training, a deep network’s training loss keeps falling, but validation loss begins to rise. What phenomenon does this pattern indicate?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Pay close attention—this one bites people in production all the time. Training loss going down while validation loss goes up means the model is memorizing the homework and bombing the pop quiz. That's overfitting (some folks say overlearning—same idea). Vanishing gradients? Learning gets stuck. Underfitting? Bad on train and val. Exploding gradients? Training blows up. Think of a student who only crams last year's answer key. Exam trap: don't pick vanishing gradients just because "something's wrong with training." Look at the two curves. Land it: train↓ val↑ = overfitting. Early stop, regularize, get more data.
Full explanation below image
Full Explanation
Generalization is the gap between performance on data the model has optimized against and performance on unseen data drawn from the same distribution. When training loss decreases while validation loss increases, the optimizer is continuing to reduce empirical risk on the training set in ways that harm held-out risk. That pattern is the textbook definition of overfitting (sometimes termed overlearning in older or informal materials): excess capacity, prolonged training, label noise, or insufficient regularization allow the model to fit idiosyncrasies and memorized details that do not transfer to new samples.
The vanishing gradient problem occurs when backpropagated gradients shrink toward zero through many layers or saturated activations, preventing effective weight updates. Symptoms include stalled or extremely slow decrease in training loss—not a healthy drop in training loss paired with rising validation loss. Underfitting is the opposite pathology: both training and validation metrics remain poor because the hypothesis class is too limited, features are weak, the learning rate is too low, or training stopped too early. Exploding gradients produce unstable optimization with extremely large parameter updates, oscillating or diverging training loss, and numerical overflow; they are diagnosed by watching gradient norms and fixed with clipping, better initialization, residual connections, or lower learning rates—not by reading the classic train–val divergence curve alone.
Mitigations for overfitting include early stopping on validation metrics, weight decay, dropout, data augmentation, gathering more diverse labeled data, simplifying the model architecture, and using cross-validation for honest model selection. Learning curves that plot train and validation loss side by side make the crossover point obvious. Always keep a final test set untouched during iterative tuning so you do not overfit the validation set itself. Memory aid: "Train happier, val sadder → overfit." On exam scenarios describing falling training loss and rising validation loss, select overfitting/overlearning, not vanishing gradients, exploding gradients, or underfitting.