To combat overfitting in a deep neural network, an engineer decides to implement a regularization technique that randomly deactivates a specified percentage of neurons during each training step. What is this technique called?

Dropout is a widely used regularization technique designed to prevent overfitting in deep neural networks. During training, dropout randomly deactivates a pre-defined fraction of neurons, along with their incoming and outgoing connections, at each update step. This forces the neural network to learn redundant representations and prevents co-adaptation of features, where certain neurons rely heavily on the outputs of specific neighboring neurons. During inference, including evaluation and testing, dropout is turned off, and all neurons are active; however, their weights are scaled down by the dropout rate to ensure the expected values remain consistent with the training phase.

The batch normalization option is incorrect because batch normalization standardizes the inputs to a layer for each mini-batch. While it can have some weak regularizing side effects, its primary purpose is to stabilize and accelerate training. The cross-validation option is incorrect because cross-validation is an evaluation methodology used to estimate how well a model generalizes to unseen data by partitioning the dataset, not a regularization technique applied to network layers during training. The early stopping option is incorrect because early stopping is a regularization method that monitors validation loss and halts the entire training process when performance stops improving, rather than modifying the active neurons within layers.