While monitoring the training progress of a deep neural network, you notice that the validation loss and training accuracy are showing highly erratic, unstable fluctuations from epoch to epoch. Which two factors are most likely contributing to this unstable training behavior? (Select two)
Select all correct answers, then click Submit.
Short Explanation and Infographic
If you're watching your training loss bounce up and down like a roller coaster, something is seriously wrong with your setup. It's usually caused by one of two culprits. First, your learning rate might be changing too wildly. Think of the learning rate like your stride when walking down a mountain. If you take giant, random leaps, you'll overshoot the path and tumble down the other side. Second, check your data prep. If your preprocessing and data augmentation are inconsistent—meaning the model sees wildly different image crops, normalization levels, or formats from batch to batch—the network gets confused and starts guessing, causing performance to fluctuate. Keep your learning rate under control and make sure your data prep is rock-solid and consistent.
Full explanation below image
Full Explanation
Erratic fluctuations in loss and accuracy during the training of deep learning models point to optimization instability or data pipeline inconsistencies. The learning rate is one of the most critical hyperparameters in gradient descent, determining the step size taken toward the local minimum of the loss function. If the learning rate is set too high or if the learning rate scheduler changes the step size too aggressively without proper decay, the optimizer will overshoot the optimal weight values. This causes the loss to bounce up and down rather than converging smoothly.
The second primary source of instability is the data pipeline. Deep learning models are highly sensitive to the scale, distribution, and format of their input features. If the data preprocessing steps (such as image normalization, resizing, or text tokenization) are applied inconsistently across different batches, or if data augmentation techniques are too extreme and run without proper constraints, the model is forced to adapt to fundamentally different data distributions at each step. This inconsistency prevents the model weights from settling, leading to fluctuating metrics.
Conversely, maintaining a fixed batch size provides training stability. Shuffling and distributing data uniformly across epochs is a standard best practice that prevents the model from memorizing order bias, thereby reducing metric fluctuations. Finally, running workloads on a single GPU vs. multiple GPUs changes the training speed and execution mechanics, but it does not inherently introduce algorithmic fluctuations in the training loss curves.