A model's loss oscillates wildly and never settles down, even growing over time. Gradients are correct. What hyperparameter is most likely set too high?
Select an answer to reveal the explanation.
Short Explanation
Wild oscillation and a loss that actually grows over time instead of settling down is the classic fingerprint of a learning rate set too high. The model keeps overshooting the minimum on every single update — like slamming on the gas and blowing straight past your parking spot every time you try to pull in. That's answer D. Dropout rate being high would hurt final accuracy and make training noisier in a different way, but it wouldn't cause this kind of runaway divergence. Too large a batch size actually tends to make the gradient estimate MORE stable, not less, so that's not your culprit either. And too many epochs just means training runs longer — it doesn't, by itself, cause the loss to diverge. When you see this bouncing, exploding pattern, drop the learning rate first.
Full Explanation
When the learning rate is set too high, each weight update overshoots the direction indicated by the gradient by such a large margin that the model repeatedly jumps past the minimum of the loss surface instead of converging toward it. This produces the classic symptom described: the loss oscillates violently from step to step and, in severe cases, can grow without bound (diverge) rather than decrease, because each update pushes the weights further from a good solution instead of closer to it.
The first distractor, dropout rate, controls how many neuron activations are randomly zeroed during training as a regularization technique; setting it too high can degrade the model's capacity to fit the data and increase noise in individual training steps, but it does not typically cause the loss to diverge in the runaway fashion described. The second distractor, batch size, affects the stability of the gradient estimate — a larger batch size actually produces a more stable, less noisy gradient (closer to full-batch behavior), the opposite of causing wild oscillation; an excessively small batch size can add noise, but the described symptom of unbounded growth is a hallmark of learning rate, not batch size. The third distractor, number of training epochs, controls how many passes are made over the data; running too many epochs risks overfitting (seen as a train/validation loss gap), not the specific oscillating-and-diverging behavior within the loss curve itself.
The standard remedy is to reduce the learning rate, often by an order of magnitude, and observe whether the loss begins to decrease smoothly. Additional mitigations include gradient clipping (capping the gradient norm to prevent any single update from being too large), using a learning-rate warmup phase to ease into training gradually, or switching to an adaptive optimizer like Adam, which can partially compensate for a suboptimal fixed learning rate by adjusting per-parameter step sizes based on gradient history.