A team wants a regularization method that can push some of their model's weights all the way to exactly zero, effectively removing those features from the model. Which comparison correctly describes the difference between L1 and L2 regularization for this goal?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal with these two: they both fight overfitting by penalizing big weights, but they punish them differently, and that difference matters a lot if you actually want feature selection. L1 adds a penalty based on the absolute value of the weights, and that penalty has a sharp corner at zero — mathematically, that sharp corner is exactly what lets gradient descent push a weight all the way down to precisely zero and keep it there. That gives you a sparse model where some features are functionally dropped out entirely. L2 adds a penalty based on the squared value of the weights, which is smooth everywhere, so it keeps shrinking weights toward zero but almost never actually lands exactly on zero — it just makes everything small. So the correct answer is C. The other options get it backwards or claim no effect at all, which just isn't how either penalty works.
Full explanation below image
Full Explanation
L1 regularization (lasso) adds a penalty term proportional to the sum of the absolute values of the weights to the loss function. Because the absolute value function has a non-differentiable 'kink' at zero, the effective gradient of this penalty exerts a constant pull toward zero regardless of how small a weight already is, which is strong enough to drive certain weights to exactly zero during optimization. This produces sparse models where irrelevant or redundant features effectively drop out of the model entirely, which is useful both for implicit feature selection and for producing a more interpretable, compact model.
L2 regularization (ridge) instead adds a penalty proportional to the sum of the squared weights. Because this penalty is smooth and differentiable everywhere, its gradient shrinks proportionally to the weight's own magnitude — as a weight gets smaller, the pull toward zero also gets smaller, asymptotically approaching but essentially never reaching exactly zero. The result is that L2 shrinks all weights together toward small values without eliminating any of them outright, generally producing better-conditioned, more stable models when many features carry some genuinely useful signal.
The first distractor is incorrect because it reverses the actual behavior: L2's smooth, squared penalty is the one that does not zero out weights, while L1's absolute-value penalty is the one capable of exact zeroing. The second distractor is incorrect because the two penalties behave differently by design — if they zeroed weights at the same rate, there would be no practical reason to choose one over the other for feature selection. The fourth distractor is incorrect because both penalties do meaningfully affect weight magnitude; that is the entire purpose of adding a regularization term to the loss function in the first place — without some pull toward smaller weights, unconstrained optimization can let weights grow arbitrarily large and overfit the training data.
A useful memory aid: 'L1 for lasso, lasso for less' — L1 can lasso in and eliminate weights entirely, giving sparse, feature-selecting models, while L2's ridge penalty rounds all the peaks down smoothly without eliminating any of them.