A team is tuning a gradient-boosted classifier and wants a systematic way to try every combination of learning rate, tree depth, and subsample ratio from a predefined set. Which technique is designed for that exhaustive hyperparameter exploration?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Okay, let's dive in. Your boss says, "Pick the best learning rate and depth—don't just guess." That's what grid search is for. You hand it a menu of hyperparameter values—learning rates, max depths, whatever—and it tries every combo on that menu, scores each one (usually with cross-validation), and points you at the winner. Think of it like testing every pizza topping pair on the menu instead of closing your eyes and hoping. PCA? That's about features, not knobs. Architecture search builds new blueprints. Training itself just fits weights. Exam trap: people mix up "tuning hyperparameters" with "training parameters." Remember—grid search is brute-force hyperparameter search over a defined grid. Got it? Sweet. Let's keep rolling.
Full explanation below image
Full Explanation
Grid search is a classical hyperparameter optimization strategy: the practitioner defines a discrete grid of candidate values for each hyperparameter of interest, and the search procedure trains and evaluates a model for every Cartesian product of those values. Performance is typically measured with cross-validation so that selection is not based on a single lucky train–test split. The configuration with the best validation score is retained for final evaluation on held-out data.
This matches the scenario of exhaustively combining learning rate, tree depth, and subsample ratio from a specified set. The method is transparent and reproducible, which is why it remains common for moderate grids and tabular models such as gradient boosting or support vector machines. Its main limitation is computational cost: as the number of hyperparameters or candidate values grows, the number of trials multiplies, so random search or Bayesian optimization is often preferred for large spaces. That cost trade-off does not change the purpose of grid search when an exhaustive discrete search is desired.
Principal component analysis is incorrect because it transforms or reduces input features to capture variance; it does not choose learning rates or tree depths. Neural architecture search targets structural design decisions—layers, connectivity, operations—rather than tuning a fixed model family's hyperparameters. Mini-batch gradient descent is an optimization algorithm for learning parameters (weights) given a loss function; those parameters are distinct from hyperparameters set before training. A useful memory aid is: parameters are learned from data; hyperparameters are chosen by search or judgment. Grid search belongs firmly in the second category and is often packaged inside scikit-learn-style cross-validation loops so preprocessing and model fitting stay consistent across folds. Best practice is to define a sensible, not infinite, grid; score with a metric aligned to business risk; and only after selection report test-set performance once to avoid optimistic bias from repeated peeking at the test set.