When is it most appropriate to apply early stopping during training?
Select an answer to reveal the explanation.
Short Explanation
Early stopping earns its name because it steps in right when things are about to go sideways: the moment validation loss starts creeping back up while training loss is still happily dropping. That rising validation loss is the tell-tale sign of overfitting — the model's starting to memorize the training set instead of learning something that generalizes. So you stop there, or roll back to the best checkpoint, before things get worse. That's answer B. Stopping as soon as training loss first starts decreasing would kill training practically at the starting line, before the model's learned anything meaningful. Waiting for the absolute maximum epoch count defeats the entire purpose of early stopping — that's just... training to the end, no early exit involved. And a learning rate schedule finishing its first decay step has nothing to do with overfitting signals — that's an unrelated part of the training process.
Full Explanation
Early stopping is a regularization technique in which training is halted once a monitored metric — most commonly validation loss — stops improving or begins to worsen, even though the training loss may still be decreasing. The appropriate trigger point is precisely when validation loss starts rising while training loss continues to fall, since this divergence is the classic signature of overfitting: the model is continuing to reduce error on the training set specifically, at the cost of generalization to unseen data. Early stopping implementations typically track the best validation performance seen so far, save a checkpoint of the model at that point, and allow a certain number of additional epochs without improvement (a "patience" window) before actually terminating training and restoring the best-performing checkpoint.
The first distractor is incorrect because stopping the moment training loss first begins to decrease would end training almost immediately, before the model has had any meaningful opportunity to learn useful patterns from the data — this is far too early and has nothing to do with the overfitting signal early stopping is meant to catch. The third distractor is incorrect because waiting until the absolute maximum number of epochs has been reached before stopping defeats the entire purpose of early stopping, which exists specifically to end training before that maximum if overfitting is detected earlier; training all the way to a fixed epoch count regardless of validation performance is the scenario early stopping is designed to avoid. The fourth distractor is incorrect because the completion of a learning-rate decay step is an unrelated event in the training schedule that has no direct bearing on whether the model is overfitting; a learning-rate schedule and early stopping are separate mechanisms that can be used together but are triggered by different signals.
Early stopping is popular because it is simple to implement, requires no architectural changes, and directly optimizes for generalization performance rather than training performance. It is often used alongside other regularization techniques such as dropout and L2 weight decay, and requires a held-out validation set that is distinct from the training set so that the stopping decision reflects genuine generalization performance rather than further fitting the training data.