During training, the training loss keeps falling steadily, but the validation loss starts rising after a certain epoch. What does this pattern indicate?
Select an answer to reveal the explanation.
Short Explanation and Infographic
This is the textbook signature of overfitting, plain and simple. Training loss keeps dropping because the model is getting better and better at memorizing the exact examples it's seeing over and over — quirks, noise, and all. But validation loss climbing back up tells you those memorized quirks don't generalize to data the model hasn't seen. The model's basically becoming a savant at the training set and worse at everything else. That's answer B. A too-high learning rate would usually show up as instability or oscillation in BOTH curves, not this specific fall-then-rise divergence pattern. Bad shuffling would show up as a weird or biased validation performance from the very start of training, not a gradual rise partway through. And batch size being too large is a hardware/memory concern, not something that produces this train/val divergence pattern. When you see the gap widen like this, think regularization, early stopping, or more data.
Full explanation below image
Full Explanation
The pattern described — training loss continuing to decrease while validation loss begins increasing after some point in training — is the classic signature of overfitting. As training continues past a certain point, the model starts to fit not just the underlying signal in the training data but also its noise and idiosyncrasies, effectively memorizing specifics of the training set that do not generalize to unseen data. This causes performance on the held-out validation set to degrade even as the model appears to be improving according to the training loss alone, creating a widening gap between the two curves.
The first distractor, a learning rate that's too high, typically produces a loss curve that oscillates or fails to decrease smoothly (or even diverges) on the training set itself, rather than the specific pattern of steadily falling training loss alongside rising validation loss; an unstable learning rate would usually be visible in the training curve, not exclusively in the validation curve. The second distractor, unshuffled train/validation splits, would tend to produce an unusual or biased validation performance from very early in training (for example, if the validation set happens to contain a systematically different distribution of examples), rather than a gradual divergence that emerges partway through an otherwise normal-looking training run. The third distractor, an oversized batch size, is primarily a memory and hardware-capacity concern (potentially causing out-of-memory errors or slower per-step computation); it does not, by itself, cause the described train/validation loss divergence.
Common remedies for this overfitting pattern include early stopping (halting training at or restoring the model from the epoch where validation loss was lowest), adding or strengthening regularization (L1/L2 weight penalties, dropout), gathering more training data, reducing model capacity (fewer parameters or layers), and applying data augmentation to expose the model to more varied examples. Monitoring the gap between training and validation loss throughout training — rather than looking at either curve in isolation — is the standard diagnostic practice for catching overfitting as early as possible.