When training a neural network for house price prediction, how does the optimization algorithm (like Gradient Descent) know whether its current weight settings are doing a good job or a terrible job, and in which direction it should adjust those weights?
Select an answer to reveal the explanation.
Short Explanation
Think of the cost function as your scorecard. It tells you how far off the model's predictions are from the real house prices, so the optimizer knows whether to nudge the weights up or down. Without it, you're just guessing in the dark.
Full Explanation
In machine learning and deep learning, a loss function (often used interchangeably with cost function, though technically a loss function computes error for a single training example while a cost function averages it over the entire dataset) is a mathematical formula that measures the discrepancy between the model's predictions and the actual ground truth labels.
The primary purpose of the cost function is to provide a single quantitative value representing the model's current performance. This value serves as the objective function that the optimization algorithm, such as Gradient Descent or Adam, seeks to minimize. During training, the optimizer calculates the gradients of the cost function with respect to the model's weights. These gradients indicate the direction and magnitude of changes needed to reduce the overall error. Popular examples of cost functions include Mean Squared Error for regression tasks and Binary or Categorical Cross-Entropy for classification tasks.
The incorrect choices do not answer the question. Using an activation function to introduce non-linear mapping helps the network learn complex non-linear decision boundaries, but it does not measure prediction error. Measuring the learning rate to dynamically scale the weight updates describes a hyperparameter that controls the step size the optimizer takes during gradient descent; it does not compute the prediction error itself. Evaluating a regularization function to penalize the complexity of the network's architecture adds a penalty to the cost function to discourage overfitting, such as L-one or L-two regularization, but it is not the primary metric measuring baseline prediction error against actual target data.
Therefore, calculating the cost or loss function, which quantifies the error between the model's predictions and the actual target values, is the component that tells the optimizer how well the current weights are doing and which direction to adjust them.