While training a deep neural network, you observe that the training loss continuously decreases to a near-zero value, but the validation loss begins to rise sharply after a certain epoch. Which approach is most appropriate to resolve this issue?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: this is a classic textbook case of overfitting. Your training loss is dropping like a rock, which means your model is memorizing the training data perfectly. But then you look at the validation loss, and it's climbing. That means when the model sees new data it hasn't memorized, it completely panics. Think of it like a student who memorized the exact questions on the study guide but fails the actual test because the questions were worded differently. To fix this, you need to make the model's life a little harder during training so it's forced to learn general patterns instead of memorizing. That's what regularization does. Using dropout randomly shuts off neurons, and L1/L2 regularization penalizes massive weights. You definitely don't want to make the model more complex or run it longer—that will just make the overfitting worse!
Full explanation below image
Full Explanation
The divergence between a low training loss and a high validation loss is the classic hallmark of overfitting (high variance). Overfitting occurs when a machine learning model learns the training data too well, capturing noise, random fluctuations, and highly specific details that do not generalize to unseen validation or test data. To mitigate overfitting, several strategies can be employed, primarily centered around regularization and reducing effective model capacity: 1) Regularization (L1/L2): Adds a penalty term to the loss function based on the magnitude of the model weights. L1 (Lasso) encourages sparsity by driving some weights to zero, while L2 (Ridge/Weight Decay) prevents any single weight from becoming too large. 2) Dropout: Randomly deactivates a fraction of neurons during each training step in a deep neural network, forcing the model to learn redundant, robust representations. 3) Early Stopping: Monitors validation loss during training and halts the process as soon as validation performance begins to degrade, preventing the model from over-optimizing on the training set. 4) Data Augmentation: Increases the diversity of the training set, making it harder for the model to memorize specific samples. The other options would worsen the issue: increasing model complexity (Option A) or increasing training epochs (Option C) gives the model even more capacity and opportunity to memorize noise. Increasing the learning rate (Option B) can cause optimization instability but does not address generalization directly.