Before training even begins, an engineer must decide on values like the learning rate, the number of layers, and the batch size. What is the general term for these kinds of values?
Select an answer to reveal the explanation.
Short Explanation and Infographic
These are hyperparameters, and the name is a good clue to what they are: settings you choose before the model ever sees a single training example, and the training process itself doesn't adjust them. Learning rate, number of layers, batch size, choice of optimizer — all decided upfront by you, often through experimentation or search, and then held fixed while the model trains. Compare that to weights, which are exactly the opposite: those start random and get learned automatically through backpropagation as training proceeds. Gradients are just the signal computed each backward pass to tell weights how to update — they're not settings at all. And activations are simply the outputs neurons produce after applying a nonlinearity during the forward pass, again nothing you configure ahead of time. Keep it simple: hyperparameters are chosen, weights are learned.
Full explanation below image
Full Explanation
The correct answer is hyperparameter, a term that specifically denotes any configuration value set before the learning process starts and not adjusted by the training algorithm itself. Learning rate, number of hidden layers, number of neurons per layer, batch size, choice of optimizer, regularization strength, and number of training epochs are all classic examples — an engineer must pick these values, often through manual experimentation, grid search, random search, or more advanced techniques like Bayesian optimization, and then the actual parameters of the model are learned given those fixed choices. This distinction matters immensely in practice: getting hyperparameters right (frequently guided by performance on a validation set) can be the difference between a model that converges beautifully and one that never learns at all or badly overfits. The first distractor, weights, describes the actual learned parameters of the network — the numbers inside each neuron's connections that are initialized (often randomly) and then updated automatically via gradient descent and backpropagation during training; weights are precisely what hyperparameters are not, since weights are learned from data rather than chosen beforehand. The second distractor, gradients, refers to the partial derivatives of the loss function with respect to the weights, computed fresh at every backward pass to indicate the direction and magnitude of weight updates — gradients are a transient computational byproduct of training, not a configuration decision made in advance, and they change constantly throughout the training process rather than staying fixed. The third distractor, activations, describes the output values produced by neurons after a nonlinear function (like ReLU or sigmoid) is applied during the forward pass — these are runtime outputs of the network's computation for a specific input, not upfront settings chosen by the engineer, and they change with every new input the network processes. A clean mnemonic: hyperparameters are the knobs you turn before the experiment starts, weights are what the experiment itself discovers, gradients are the instructions for how to adjust those weights, and activations are just intermediate signals flowing through the network during a single pass.