During training of a deep network, what is the main purpose of a dropout layer?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal with dropout: during training, you randomly bench some players each play so the team can't rely on one superstar. Neurons stop co-adapting into brittle little cliques, and the network learns more robust patterns. At inference you use the full team (with scaling). Exam trap: people mix dropout with "add more neurons," "crank the learning rate," or batch norm. Different tools. If train accuracy is sky-high and test is soft, dropout is one of the classic regularizers you reach for—along with weight decay and more data. Think noisy practice, cleaner game-day performance. Got it? Sweet.
Full explanation below image
Full Explanation
Dropout is a regularization method introduced to combat overfitting in large neural networks. At each training step (or per sample, depending on implementation), each unit in a dropout layer is kept with probability p and otherwise set to zero. This stochastic masking prevents co-adaptation—units cannot rely on particular partners always being present—so the network learns distributed, redundant features. At test time, units are typically all active and outputs are scaled (or inverted dropout is used during training) so that expected activations match.
Dropout is not a capacity-expansion technique; it does not permanently add neurons. Learning-rate control belongs to optimizers and schedulers. Feature re-centering and re-scaling across a batch is the role of batch normalization (and related normalizers), which address internal covariate shift and optimization dynamics rather than randomly dropping units—though both can appear together in modern architectures.
Practically, dropout rates are treated as hyperparameters (common values historically around 0.1–0.5 depending on layer type). Variants include DropConnect, spatial dropout for CNNs, and attention dropout in transformers. Because dropout injects noise only during training, a large train–test gap that shrinks after adding dropout is a common diagnostic story. Underlying principle: stochastic unit masking is a regularizer that improves robustness by reducing co-adaptation, not a way to grow the network or control step size. Best practice tunes the keep probability on validation data, disables dropout semantics correctly at inference, and pairs dropout with other capacity controls when needed. Memory aid: dropout means randomly silence units while learning to reduce overfitting; batch norm means normalize activations; learning-rate schedule means how big the steps are. Keep those jobs separate on the exam.