While tracking your deep learning model's metrics over successive training epochs, you observe a frustrating trend: the model's accuracy on the validation dataset improves at first, hits a plateau, and then starts steadily dropping as training continues. Which two factors are most likely responsible for this behavior? (Choose two)
Select all correct answers, then click Submit.
Short Explanation and Infographic
Check this out: when you're training a network, you want that accuracy line on your validation chart to keep climbing and then level off nicely. But if it starts diving back down, you're likely dealing with one of two classic headaches. First, overfitting. Think of it like a student who memorizes the exact questions on the practice test word-for-word but fails the actual exam because they didn't learn the concepts. The model gets super smart on the training data but completely flops on new data. Second, a learning rate that's way too high. Think of this like driving a car at 100 miles an hour trying to park in a tight spot—you're just going to keep overshoot-overshoot-overshooting the spot and crashing into the curb. If your step size is too big, the optimizer bounces around and can't settle down, causing instability. Trust me, finding that sweet spot for your learning rate and keeping your model from memorizing the noise are key to passing this test and getting your models to perform in production. Got it? Sweet. Let's keep rolling.
Full explanation below image
Full Explanation
The behavior of validation accuracy improving, plateauing, and then declining is a classic symptom of two distinct machine learning challenges: overfitting and learning rate instability. Overfitting (Option B) occurs when a model learns the training data too well, including its noise and outliers, rather than learning the underlying distribution. As training progresses beyond the plateau, the model continues to optimize its parameters for the training set, causing its ability to generalize to unseen validation data to deteriorate, which drops validation accuracy. A high learning rate (Option A) causes the optimization algorithm (like Gradient Descent) to take steps that are too large. While it might initially make rapid progress toward a local minimum, it will eventually overshoot the optimal weights and bounce back and forth, preventing convergence and causing instability or divergence in performance. Cross-validation (Option C) is an evaluation methodology, not a training dynamic that degrades model performance. Inadequate dataset size (Option D) typically leads to early underfitting or immediate overfitting, but does not explain a steady rise and decline behavior caused by optimizer dynamics. Proper regularization (Option E) is designed specifically to prevent overfitting, so its correct application would stabilize validation accuracy rather than cause it to decline.