In machine learning workflows, what is a hyperparameter?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Boss walks in: "What learning rate and depth did we pick?" Those knobs — set before (or outside) the weight-update loop — are hyperparameters. Learning rate, batch size, number of trees, regularization strength, network width… you choose them, then training learns the actual parameters. Think of it like oven temperature versus the chemical changes in the cake. You set the temp; the cake reacts. Exam trap: calling weights hyperparameters. Nope — weights are parameters. Another trap: calling accuracy a hyperparameter. That's a metric. And hyperparameters aren't "auto-magic numbers with no selection" — we set them or run grid/random/Bayesian search. Remember: hyper = settings for learning; parameters = what learning finds. You'll see this split constantly on the test.
Full explanation below image
Full Explanation
Machine learning distinguishes parameters from hyperparameters. Parameters are internal values estimated by the training procedure to minimize (or maximize) an objective—for example neural network weights and biases, linear regression coefficients, or leaf values in some tree ensembles. Hyperparameters are external configuration choices that define the model class, optimization setup, or regularization before (or outside) that inner learning loop.
Typical hyperparameters include learning rate, momentum, batch size, number of epochs, network depth and width, dropout rates, L2 penalty strength, number of neighbors in k-NN, maximum tree depth, and the k in k-means. Because they are not learned by ordinary gradient steps on the training loss in the same way as weights, practitioners select them using validation performance via grid search, random search, Bayesian optimization, or bandit-based methods such as Hyperband. Nested cross-validation helps avoid overfitting the validation set during this outer search.
Calling learned weights hyperparameters confuses the two layers of the workflow. Claiming hyperparameters are values the algorithm invents with no selection process ignores that someone or some outer loop must specify them. Evaluation metrics (accuracy, MAE, AUC) quantify quality after or during training; they guide hyperparameter search but are not themselves hyperparameters of the model.
Practical tip: document hyperparameter choices for reproducibility, and freeze them before final test evaluation. On exams, any option describing "set before training" or "controls learning but is not a learned weight" is the hyperparameter definition; options about learned coefficients or scoring metrics are distractors. Memory aid: parameters are fit during training; hyperparameters are picked beforehand or via an outer validation search to make that fitting work well. Confusing the two layers is one of the most common beginner mistakes in applied machine learning.