During the training phase of a deep neural network, a machine learning engineer notices that the training loss and accuracy metrics are exhibiting highly unstable, oscillating patterns from epoch to epoch. Which of the following factors are most likely to introduce this type of performance instability during model training? (Choose two)
Select all correct answers, then click Submit.
Short Explanation and Infographic
Pay close attention here, because this one bites people in the lab all the time. You start training a model, you look at your loss curve, and instead of a nice, smooth downward slope, it looks like a crazy roller coaster going up and down. What's going on? Two big culprits. First is the learning rate. Think of the learning rate like the stride of a person walking down a mountain. If it's too big, they'll overshoot the valley floor and end up climbing the opposite peak—that's your model overshooting the optimal weights and causing metrics to bounce around. Second is your data preprocessing. If you're not preprocessing your inputs consistently—say, scaling some images differently or changing color spaces mid-run—you're basically feeding the model garbage data. It's trying to hit a moving target! Keeping a fixed batch size and shuffling your data actually stabilizes training. Trust me on this, keep your preprocessing rock-solid and tune your learning rate carefully. Got it? Let's keep rolling.
Full explanation below image
Full Explanation
Unstable training metrics (such as erratic swings in training loss or validation accuracy) indicate that the model is struggling to converge during gradient descent. Identifying the root causes of these fluctuations is critical for successful model training. - Learning Rate Adjustments (Option A): The learning rate is a hyperparameter that controls how much to adjust the model's weights in response to the estimated error gradient. If the learning rate is set too high, the optimization algorithm can overshoot the local minima, causing the loss to diverge or oscillate wildly. While learning rate schedules (like decay or cyclic learning rates) are standard practices, abrupt or excessive changes to the learning rate during training can lead to immediate spikes or drops in performance metrics. - Data Preprocessing Variation (Option D): Machine learning models assume that the training data distribution remains relatively stable. If the preprocessing pipeline is inconsistent (e.g., varying scaling ranges, inconsistent normalization, or changing augmentation pipelines between batches without control), the model receives inputs with shifting distributions. This forces the model to constantly adjust its weights to conflicting data representations, preventing stable learning and causing metrics to fluctuate.
Let's analyze why the other options do not cause metric fluctuations: - Option B is incorrect because using a fixed batch size is standard practice and provides a consistent gradient estimation size, which stabilizes the optimization path. - Option C is incorrect because ensuring a uniform distribution of data across epochs (often achieved via dataset shuffling) is a best practice. It prevents the model from learning order-based biases, thereby smoothing convergence rather than causing fluctuations. - Option E is incorrect because training on a single GPU does not inherently cause metric instability; it simply limits parallel throughput and execution speed.
Therefore, Options A and D are the primary contributors to training fluctuations.