A team notices their deep network performs very well on training data but poorly on validation data. They add a dropout layer to help. What does a dropout layer actually do?
Select an answer to reveal the explanation.
Short Explanation
That gap between great training performance and poor validation performance is the classic sign of overfitting — the network is memorizing training examples instead of learning general patterns. Dropout fights this by randomly switching off a fraction of neurons on every training step, forcing the remaining neurons to not rely too heavily on any one path through the network. That's the fourth option, and it's correct. It's not about permanently deleting neurons from the architecture — dropout is temporary and random each step, and at inference time all neurons are back and active. It's not a learning-rate trick either; adjusting the learning rate based on validation plateaus is a different technique entirely. And it's not normalization — zero-mean, unit-variance scaling is what batch normalization does, a separate tool with a separate job.
Full Explanation
Dropout is a regularization technique designed to combat overfitting, which is exactly the symptom described: strong training performance paired with weaker validation performance, indicating the model has learned patterns specific to the training set rather than patterns that generalize. During each training step, a dropout layer randomly sets a specified fraction of neuron activations (commonly 20 to 50 percent) to zero, and this random subset changes on every forward pass. Because no neuron can rely on any specific set of other neurons being present, the network is forced to learn more redundant, distributed representations rather than co-adapting narrowly around specific unit combinations, which improves generalization to unseen data. Critically, dropout is only applied during training; at inference time all neurons remain active, typically with their outputs scaled to account for the fact that dropout was active during training. This makes the fourth option correct. The first option, describing dropout as permanently removing a fixed set of neurons before training begins, is incorrect because dropout's masking is random and changes every training step, and it never permanently alters the architecture; permanently removing neurons would instead be a form of network pruning, a distinct technique typically applied after training for model compression. The second option, describing dropout as temporarily raising the learning rate when validation loss plateaus, actually describes a learning-rate scheduling strategy (such as reducing on plateau, which typically lowers rather than raises the rate), an entirely separate optimization concern unrelated to neuron activation masking. The third option, describing dropout as normalizing activations to zero mean and unit variance, describes batch normalization (or layer normalization), a different regularization and training-stabilization technique that rescales activation statistics rather than randomly zeroing them out. A helpful memory aid: think of dropout as randomly benching different players from a sports team at every practice session, so the team learns to function well regardless of which specific players are on the field, rather than becoming overly dependent on one star player who might not always be present.