While tracking your deep learning model's metrics over successive training epochs, you observe a frustrating trend: the model's accuracy on the validation dataset improves at first, hits a plateau, and then starts steadily dropping as training continues. Which two factors are most likely responsible for this behavior? (Choose two)
Select all correct answers, then click Submit.
Short Explanation
Check this out: when validation accuracy climbs, plateaus, then drops, you're usually dealing with overfitting or a learning rate that's too high. The model starts memorizing noise instead of generalizing, or the optimizer overshoots and can't settle. Dial in a better learning rate and control overfitting to keep validation performance stable.
Full Explanation
The behavior of validation accuracy improving, plateauing, and then declining is a classic symptom of two distinct machine learning challenges: overfitting and learning rate instability. Overfitting occurs when a model learns the training data too well, including its noise and outliers, rather than learning the underlying distribution. As training progresses beyond the plateau, the model continues to optimize its parameters for the training set, causing its ability to generalize to unseen validation data to deteriorate, which drops validation accuracy. The learning rate being configured too high causes the optimization algorithm, such as gradient descent, to take steps that are too large. While it might initially make rapid progress toward a minimum, often a local minimum in practice and ideally a global minimum, it will eventually overshoot the optimal weights and bounce back and forth, preventing the optimizer from settling into a stable global minimum and causing instability or divergence in performance. Dividing the dataset into multiple subsets through k-fold cross-validation is an evaluation methodology, not a training dynamic that degrades model performance. Insufficient training samples typically lead to early underfitting or immediate overfitting, but do not explain a steady rise and decline behavior caused by optimizer dynamics. Regularization techniques such as weight decay operating correctly are designed specifically to prevent overfitting, so their correct application would stabilize validation accuracy rather than cause it to decline.