What does the process of hyperparameter tuning involve?
Select an answer to reveal the explanation.
Short Explanation
Hyperparameter tuning is the process of searching for the best values for settings you choose before training even starts — things like learning rate, batch size, number of layers, or dropout rate. These aren't learned by the model through gradient descent; you, the practitioner, set them, often through trial and error, grid search, or automated search strategies, then train and see how it performs. That's answer B. It's different from adjusting weights via backpropagation, that's the model's parameters being learned automatically, not hyperparameters chosen beforehand. It's also not about cleaning and labeling your dataset, that's data preparation, a separate step. And there's no automatic runtime process that magically picks your activation function for the lowest loss — that's a hyperparameter choice, not something the network selects mid-training. Hyperparameter tuning is squarely about searching over practitioner-chosen settings.
Full Explanation
Hyperparameter tuning refers to the process of searching for the best combination of settings that govern how a model is trained, but which are not themselves learned from the data via gradient descent. Common hyperparameters include the learning rate, batch size, number of layers, number of neurons per layer, choice of optimizer, regularization strength, and dropout rate. Because the relationship between these settings and final model performance is often complex and non-obvious, practitioners commonly use systematic search strategies such as grid search, random search, or more sophisticated methods like Bayesian optimization, to explore the hyperparameter space and identify a configuration that yields strong performance on a validation set.
The first distractor describes the training process itself, specifically how a model's weights (its learned parameters) are updated via backpropagation and an optimizer; weights are learned automatically from data during training, which is a fundamentally different process from hyperparameter tuning, where a human (or an automated search algorithm) sets values before or between training runs. The second distractor describes data preprocessing and labeling, an important but separate stage of the machine learning pipeline concerned with preparing the dataset itself, not with configuring the model's training settings. The third distractor describes a fictional automatic process where the network would select its own activation function at runtime to minimize loss; in practice, the activation function is a hyperparameter chosen by the practitioner (or through hyperparameter search) prior to training and remains fixed throughout a given training run — it is not dynamically selected by the model as training proceeds.
A useful memory aid: parameters (like weights and biases) are what the model learns from data during training, while hyperparameters (like learning rate, batch size, and dropout rate) are the dials a practitioner sets beforehand, and hyperparameter tuning is the systematic process of finding the best positions for those dials.