A dataset has 200 numeric features, many of which are correlated, and a team wants to compress it into a much smaller set of features while retaining as much of the original variance as possible before feeding it into a downstream model. Which technique is the standard choice for this?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: when you've got a pile of correlated features and you want to shrink them down without throwing away the important information, that's Principal Component Analysis, answer A. PCA finds new axes — the principal components — that point in the directions where your data varies the most, and it ranks them so you can keep just the top few and still capture most of the original spread. K-Means is a distractor because it's a clustering algorithm; it groups similar data points into buckets, it doesn't reduce the number of features at all. Ridge regularization is about penalizing large weights during model training to fight overfitting, not about compressing your feature space. And one-hot encoding actually increases your dimensionality by expanding a categorical variable into multiple binary columns — the opposite of what you're after here.
Full explanation below image
Full Explanation
Principal Component Analysis (PCA) is the standard technique for reducing the dimensionality of a dataset while preserving as much of its variance as possible. PCA works by computing the covariance matrix of the features, then finding its eigenvectors and eigenvalues. The eigenvectors define new, uncorrelated axes (the principal components), and the eigenvalues indicate how much variance each axis captures. By keeping only the top-k principal components — those with the largest eigenvalues — you can represent the data in far fewer dimensions while retaining the bulk of the original variance, which is especially useful when many original features are correlated and therefore redundant.
K-Means clustering is incorrect because it is an unsupervised partitioning algorithm that assigns data points to a fixed number of clusters based on distance to cluster centroids. It groups observations; it does not transform or reduce the number of features describing each observation, so it does not address the stated goal at all.
L2 (ridge) regularization is incorrect because it is a training-time technique that adds a penalty on the squared magnitude of model weights to reduce overfitting and stabilize coefficient estimates. It operates on model parameters, not on the input feature space, and does not reduce the number of input dimensions.
One-hot encoding is incorrect because it is a technique for representing categorical variables numerically, and it typically increases the number of features (one binary column per category) rather than reducing them. It solves a different problem entirely — representation of categorical data — and moves in the opposite direction from dimensionality reduction.
Memory aid: 'PCA compresses correlated columns into a few variance-ranked axes' — if the goal is fewer features while keeping the signal, PCA is almost always the intended answer among classical ML techniques.