An engineer is choosing between L1 and L2 regularization for a model and wants some of the less useful weights to be driven exactly to zero, effectively performing feature selection. Which statement correctly distinguishes the two approaches for this goal?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's a quick way to remember it: L1 has sharp corners, and sharp corners are where weights get pushed all the way to zero. L2 is round and smooth, so it just gently shrinks weights without ever fully zeroing them out. So if the engineer wants sparsity — some weights landing at exactly zero, basically doing feature selection for free — that's L1, answer B. L2 (option A reversed) does the opposite: it spreads the penalty out and shrinks everything a bit, which is great for stability but won't zero anything out. They're not identical, so option C is wrong, and it's simply false that neither can hit exactly zero — L1 does that routinely.
Full explanation below image
Full Explanation
L1 regularization adds a penalty proportional to the sum of the absolute values of the weights. Geometrically, the L1 penalty region has sharp corners along the coordinate axes, and the optimal solution under this penalty frequently lands exactly on one of those corners, which corresponds to certain weights being exactly zero. This is why L1 regularization (as in Lasso regression) is associated with sparse solutions and is often used for automatic feature selection: irrelevant or redundant features end up with zero weight and are effectively removed from the model.
L2 regularization adds a penalty proportional to the sum of the squares of the weights. Its penalty region is smooth and round (a hypersphere), and the gradient of the squared penalty shrinks toward zero as the weight approaches zero, meaning the pressure to reduce a weight further weakens as it gets small. As a result, L2 (as in Ridge regression) tends to shrink all weights toward small values but rarely forces any individual weight to be exactly zero. It is often preferred when all features are believed to contribute at least a little, and for improving numerical stability and reducing variance without eliminating features.
The first option reverses which technique produces sparsity, describing L2 as sparsity-inducing and L1 as merely shrinking — this is backwards relative to their actual mathematical behavior.
The third option is incorrect because L1 and L2 have fundamentally different penalty geometries and optimization behavior; they do not behave identically, and the set of weights driven to zero (if any) is generally different between the two.
The fourth option is incorrect because L1 specifically is well known for producing exact zeros; this is one of its most cited practical advantages over L2.
Memory aid: "L1 = Lasso = Less (sparse, some weights vanish); L2 = Ridge = Reduce-all (shrinks everything smoothly)."