A researcher training a deep network for 200 epochs notices validation loss stopped improving around epoch 60 and slowly worsened afterward. What technique would have helped avoid wasting the remaining epochs and overfitting the model?
Select an answer to reveal the explanation.
Short Explanation and Infographic
This is exactly the textbook scenario early stopping was built for. You watch validation loss during training, and once it stops improving and starts creeping upward while training loss keeps dropping, your model's tipping from learning general patterns into memorizing training noise. Early stopping means halting right around that turning point — often keeping the best validation checkpoint — instead of grinding through all 200 epochs and ending up worse than you were 140 epochs earlier. That's answer A. Batch normalization is a totally different tool — it standardizes layer activations, it doesn't watch validation loss or stop training. Label smoothing softens hard targets to reduce overconfidence, unrelated to deciding when to stop. And data augmentation fights overfitting by expanding your training set, a legitimate tool, but it doesn't address training needlessly past the point where validation performance peaked.
Full explanation below image
Full Explanation
Early stopping is a regularization technique that monitors a model's performance on a validation set during training and halts training once that validation metric (commonly validation loss) stops improving for a specified number of epochs (a 'patience' window), rather than running for a fixed, predetermined number of epochs regardless of what is happening to generalization performance. In the scenario described, validation loss stopped improving around epoch 60 and then began to slowly worsen, which is the classic signature of overfitting: the model continues reducing training loss by fitting increasingly specific patterns and noise present only in the training data, while its ability to generalize to unseen validation data degrades. Early stopping would have halted training at or shortly after epoch 60, typically restoring the model weights from the epoch with the best validation performance, avoiding 140 wasted epochs of compute and preventing the model from settling into an overfit state.
The first distractor, batch normalization, standardizes the activations within a layer to have consistent mean and variance across mini-batches, which helps stabilize and often accelerate training and can have a mild regularizing side effect, but it has no mechanism for monitoring validation performance over epochs or deciding when to halt training. The third distractor, label smoothing, replaces hard one-hot target labels with slightly softened versions, discouraging overconfidence, which can reduce overfitting somewhat, but it changes the loss computation throughout all epochs rather than detecting and responding to a validation-loss plateau, so it does not solve the wasted-epochs problem posed here. The fourth distractor, data augmentation, combats overfitting by artificially expanding the effective diversity of the training set through transformations like rotation, cropping, or noise injection, which can reduce the overfitting gap over time, but like label smoothing, it is a preventive measure applied throughout training rather than a stopping-rule mechanism reacting to an observed validation-loss inflection point.
A good memory aid: early stopping is a 'when to stop' regularizer, while batch normalization, label smoothing, and data augmentation are all 'how to train better' tools that apply throughout every epoch rather than making a halting decision. In practice, early stopping is often combined with checkpointing, saving the best-performing model snapshot on the validation set, so the deployed model reflects the best generalization point observed, not whatever state training happened to end in.