When configuring a neural network training run, you must manually define certain configuration settings—like the learning rate, the batch size, and the number of layers—before the training process even begins. What are these manual configuration settings called?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: think of hyperparameters like the knobs and dials on a high-end stereo system. You have to manually adjust them—like the bass, treble, and volume—to get the sound just right. In deep learning, you are the one setting these knobs (like learning rate, batch size, and network depth) before the model starts training. The model can't set them for you! The things the model does figure out on its own during training are called model parameters (weights and biases). Don't mix these two up, or it'll bite you on the exam.
Full explanation below image
Full Explanation
In machine learning and deep learning, a fundamental distinction is made between model parameters and hyperparameters. Hyperparameters are the external configuration settings specified by the engineer before the training process begins. They control the overall behavior of the training algorithm and the structure of the network. Key examples include the learning rate, batch size, number of epochs, activation functions, and the number of layers and hidden units. These cannot be learned directly from the training data because they define the environment in which the model learns. In contrast, model parameters are the internal variables that the model learns and updates automatically during training using optimization algorithms (such as gradient descent). These include weights and biases. Distractors like model outputs refer to the predictions made by the network (e.g., probability scores), while input variables (features) represent the data points fed into the model. Properly tuning hyperparameters is one of the most critical and time-consuming tasks in model development, often requiring techniques like grid search, random search, or Bayesian optimization.