A deep network is training unstably, with loss swinging wildly between mini-batches, and convergence is painfully slow. An engineer suggests inserting batch normalization layers between the linear layers and activations. What problem is batch normalization primarily designed to address?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Batch normalization exists because as data flows through a deep network, the distribution of each layer's inputs keeps shifting as the weights in earlier layers update — that's internal covariate shift, and it makes later layers chase a moving target every single step, which is exactly the wild, slow training you're describing. Batch norm fixes this by re-centering and re-scaling each layer's inputs (per mini-batch) so they stay in a consistent, well-behaved range, which lets you use higher learning rates and converge faster and more smoothly. It has nothing to do with shrinking the model's parameter count, it doesn't touch or replace your validation workflow, and it doesn't rewrite your loss function — it operates on the activations flowing between layers, not the objective you're optimizing against.
Full explanation below image
Full Explanation
The correct answer is that batch normalization reduces internal covariate shift — the phenomenon where the distribution of inputs to each layer keeps changing during training as the parameters of preceding layers are updated — by normalizing each mini-batch's activations to have roughly zero mean and unit variance before applying learnable scale and shift parameters. This keeps the input distribution to each layer more stable across training steps, which in turn allows for higher learning rates, faster convergence, and reduces the network's sensitivity to initialization, directly addressing the erratic, slow-converging behavior described in the scenario. The first distractor, about compressing parameter count, is incorrect because batch normalization actually adds a small number of extra learnable parameters (a scale and shift per normalized dimension) rather than reducing them; it's a normalization technique, not a compression or pruning technique, and has no relationship to model size or deployment footprint. The second distractor, about removing the need for a validation set, confuses batch normalization with an entirely separate concern — validation sets exist to guide hyperparameter tuning and detect overfitting, a role that normalizing activations inside the network does nothing to replace; you still need held-out data regardless of what normalization layers you use. The fourth distractor, about replacing the loss function, misunderstands where batch normalization operates: it transforms the activations passed between layers during the forward pass, not the loss function used to measure error at the output, and the loss function's differentiability is unaffected by whether batch norm layers exist elsewhere in the network. A helpful way to remember batch normalization's role: it keeps the 'input weather' for each layer predictable and stable, so that layer doesn't have to constantly re-adapt to shifting conditions caused by upstream weight updates, which is the core mechanism behind its stabilizing and accelerating effect on training.