Which regularization technique works by randomly setting a fraction of neuron outputs to zero during each training pass, forcing the network to avoid relying too heavily on any single neuron?
Select an answer to reveal the explanation.
Short Explanation
This one's dropout, no question. During training, dropout randomly switches off a chunk of neurons in a layer for that particular forward and backward pass — their outputs just get zeroed out, like they temporarily don't exist. Which neurons get dropped changes randomly every batch. The whole point is to stop the network from becoming overly reliant on any one specific neuron or tight little clique of neurons memorizing quirks of the training data. Instead it's forced to spread useful representations across the network more robustly, which helps it generalize better to new data. That's answer C. Batch normalization does something totally different — it normalizes layer activations to stabilize and speed up training, no random zeroing involved. Early stopping just halts training once validation performance stops improving, nothing to do with zeroing neurons. And L2 weight decay shrinks weight magnitudes smoothly through a penalty term, again a completely different mechanism than randomly dropping neuron outputs.
Full Explanation
Dropout is a regularization technique in which, during each training iteration, a randomly selected fraction of neurons in a given layer (controlled by a dropout rate hyperparameter, commonly between 0.2 and 0.5) have their outputs set to zero, effectively removing them from that particular forward and backward pass. Because a different random subset of neurons is dropped on every batch, the network cannot become overly dependent on any specific neuron or narrow co-adapted group of neurons to make its predictions. This forces the remaining active neurons to learn more robust, redundant representations that are useful on their own, which reduces overfitting and generally improves the model's ability to generalize to unseen data. At inference/test time, dropout is turned off and the layer outputs are typically scaled to account for the fact that no neurons are being dropped.
Batch normalization is incorrect because it works by normalizing the activations of a layer (typically to zero mean and unit variance, followed by a learnable scale and shift) across each mini-batch, which stabilizes and accelerates training and can have a mild regularizing side effect, but it does not involve randomly zeroing out any neuron outputs. Early stopping is incorrect because it is a regularization strategy that monitors validation performance during training and halts training once that performance stops improving or starts degrading, preventing the model from continuing to fit noise in the training data; it operates at the level of when to stop training, not by modifying individual neuron outputs during each pass. L2 weight decay is incorrect because it regularizes by adding a penalty term proportional to the sum of squared weights to the loss function, which smoothly shrinks all weight magnitudes toward zero over the course of training; it never sets any activation or weight to exactly zero at random, and it operates on the weights rather than on the neuron outputs during a forward pass.
A helpful mental model: dropout can be thought of as training a large ensemble of overlapping 'thinned' subnetworks simultaneously, since a different random subnetwork is active on every training step, and the final trained network at inference time behaves like an approximate average over all of those thinned subnetworks, which is part of why dropout is such an effective and widely used regularizer in deep networks with many parameters.