When comparing L1 and L2 weight regularization, which statement correctly describes a key difference?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal on L1 vs L2 — people flip them constantly on exams. L1 is the sparse one: it likes to zero out weights, like flipping unused feature switches off. L2 is the gentle shrinker: keeps weights small and shared, rarely exact zeros. Think of L1 as a declutter crew throwing junk out; L2 as gently turning the volume down on everything. Exam trap: "L1 = classification only," "L1 always better against overfit," or swapping absolute vs squared penalties. Remember: L1 absolute → sparsity; L2 squared → weight decay style shrinkage. When your boss wants automatic feature selection vibes, L1's the usual pointer.
Full explanation below image
Full Explanation
Weight regularization adds a penalty term to the training loss to discourage overly complex parameterizations and reduce overfitting. L1 regularization (lasso-style) adds a term proportional to the sum of absolute values of weights. The geometry of the L1 ball encourages solutions on axes, so many coefficients become exactly zero—yielding sparse models useful for feature selection and interpretability. L2 regularization (ridge or weight-decay style) adds a term proportional to the sum of squared weights, which shrinks parameters toward zero continuously; coefficients usually remain small but non-zero, spreading influence across correlated features.
The correct option states this sparsity-versus-shrinkage contrast. Elastic Net combines both when mixed behavior is desired. Neither regularizer is restricted to classification or regression; both appear in linear models, neural nets, and other estimators. Claiming L1 always superior for overfitting is false: correlated features, noise structure, and the strength of the penalty matter. Swapping absolute and squared penalties is a frequent error—memorize L1 equals absolute, L2 equals squared.
In neural networks, L2 weight decay is extremely common; L1 is used when sparsity is explicitly wanted. Always tune the regularization strength on validation data: too large underfits, too small fails to curb overparameterization. Pair regularization with adequate data, early stopping, and architecture choices rather than treating any single penalty as magic.
Underlying principle: capacity control trades training fit against generalization by constraining parameters. Best practice documents which penalty is used, tunes lambda or weight decay carefully, and interprets sparse L1 solutions with care when features are highly correlated. Memory aid: L1 leaves some weights lonely at zero; L2 lessens all weights a little. Pair that with absolute versus squared definitions and exam questions on this topic become straightforward.