While training a deep neural network, you observe that the training loss continuously decreases, but the validation loss begins to rise after a certain epoch. Which technique should you implement to resolve this issue?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: when your training loss keeps dropping but your validation loss starts climbing, your model is starting to overfit. It's essentially memorizing the training data instead of learning general patterns. The simplest way to handle this? Just stop training before it goes off the rails! This is called early stopping (sometimes translated as early shutdown). You monitor the validation loss, and if it doesn't improve for a few epochs, you halt the training run and save the best weights. So, A is the correct answer. Option B is wrong because increasing the learning rate might cause the model to overshoot and make performance worse. Option C is wrong because increasing batch size affects training dynamics and speed, but won't solve this specific overfitting trend. Option D is done before training begins, not mid-run. Stop training when it's at its peak, and you'll save time and get a better model!
Full explanation below image
Full Explanation
The divergence of training loss (decreasing) and validation loss (increasing) is a hallmark symptom of overfitting. The model is learning noise and patterns specific to the training set that do not generalize to unseen validation data. Early stopping is a regularization technique that monitors the model's loss or accuracy on an independent validation set during training. When validation performance stops improving and begins to degrade (typically measured over a window of epochs called 'patience'), the training process is programmatically terminated. The weights from the epoch that achieved the lowest validation loss are then restored.
Let's analyze the options: - Option A is correct because early stopping halts the training loop at the point of optimal generalization before overfitting deteriorates model utility. - Option B is incorrect because increasing the learning rate will cause weight updates to take larger steps, which usually increases instability and does not address overfitting. - Option C is incorrect because increasing the batch size changes the noise profile of gradient updates and alters training throughput, but it does not act as a regularizer or stop training when validation loss degrades. - Option D is incorrect because weight initialization occurs only once before the training loop starts and cannot be applied dynamically to address overfitting observed mid-training.