What does L2 regularization add to a model's loss function?
Select an answer to reveal the explanation.
Short Explanation
L2 regularization tacks an extra term onto your loss function based on the squared value of every weight in the network. Square them, sum them up, multiply by a small coefficient (often called lambda), and add that to the loss. That's answer B. Because the penalty grows with the square of the weight, big weights get punished disproportionately hard, which nudges the optimizer to keep weights small and spread out rather than letting a handful of weights blow up. Using absolute values instead of squares describes L1 regularization — related idea, different math, different behavior. A random mask zeroing out weights each step is dropout, not L2 at all. And a term that rewards bigger weights would be the exact opposite of regularization — regularization exists specifically to discourage large weights, not encourage them.
Full Explanation
L2 regularization, also known as weight decay or ridge regularization, adds a penalty term to the loss function equal to a regularization coefficient (lambda) multiplied by the sum of the squares of all the model's weights: Loss_total = Loss_original + lambda * sum(w_i^2). Because the penalty scales with the square of each weight's magnitude, the optimizer is pushed to keep weights small in aggregate during training, since large weights would otherwise incur a disproportionately large penalty. This tends to produce smoother, more evenly distributed weight values and helps reduce overfitting by discouraging the model from relying too heavily on any single feature or connection.
The first distractor describes L1 regularization, which uses the sum of the absolute values of the weights rather than their squares. L1 has a qualitatively different effect: because the penalty's derivative is constant (not proportional to the weight's magnitude), L1 regularization can push some weights all the way to exactly zero, producing sparse models — a property L2 generally does not share, since L2's gradient shrinks as the weight approaches zero, causing weights to shrink toward but rarely reach exactly zero. The second distractor describes dropout, a different regularization technique entirely that randomly zeroes a fraction of neuron activations during each forward pass at training time, rather than adding any penalty term to the loss function based on weight values. The third distractor is incorrect because regularization techniques exist specifically to constrain model complexity and reduce overfitting by discouraging large weights, not to reward or encourage them; rewarding large weights would increase overfitting risk, the opposite of regularization's purpose.
L2 regularization is mathematically equivalent, under standard SGD, to weight decay, where each update subtracts a small fraction of the current weight value in addition to the gradient-based update. It is one of the most widely used regularization techniques in deep learning because it is simple to implement, differentiable everywhere, and effective at controlling model complexity without eliminating any particular weight or feature outright, in contrast to L1's tendency toward sparsity.