What is the purpose of a cost (loss) function in training a neural network?
Select an answer to reveal the explanation.
Short Explanation
A loss function is basically the model's report card after every prediction — it measures exactly how far off the predicted output is from the actual, correct target value. A high loss means the model got it pretty wrong, a low loss means it's close, and that single number is what backpropagation and the optimizer use to figure out how to adjust the weights and do better next time. That's the first option here, plain and simple. It's not about defining how many neurons sit in a hidden layer, that's an architecture design choice you make ahead of time. It's not about choosing which optimizer to use either, those are two separate pieces that work together, loss tells you 'how wrong,' optimizer decides 'how to fix it.' And it has nothing to do with batch size, which just controls how many examples get grouped together per training step. The loss function's one job is measuring error.
Full Explanation
A cost or loss function quantifies how far a neural network's predictions deviate from the true target values for a given set of inputs. Common examples include mean squared error for regression tasks, which penalizes the squared difference between predicted and actual continuous values, and cross-entropy loss for classification tasks, which penalizes the divergence between predicted class probabilities and the true class labels. This scalar error signal is the foundation of the entire training process: backpropagation computes the gradient of the loss with respect to every weight in the network, and the optimizer uses those gradients to adjust the weights in the direction that reduces the loss. Without a well-defined loss function, there would be no quantitative target for the network to minimize, and therefore no mechanism to guide learning.
The first distractor describes an architectural hyperparameter, the number of neurons per hidden layer, which is a design decision made when constructing the network and remains fixed during a given training run; it has no relationship to how prediction error is measured. The second distractor confuses two related but distinct components of training: the loss function measures the error the model is trying to minimize, while the optimizer (such as SGD, Adam, or RMSprop) is a separate algorithm that determines how weights are updated given the gradients of that loss; choosing a loss function does not itself select or determine the optimizer, and the two can be mixed and matched. The third distractor describes batch size, a training hyperparameter that specifies how many training examples are processed together before an update is applied; this affects the stability and speed of gradient estimates but is unrelated to how the discrepancy between predictions and targets is quantified.
A helpful memory aid: the loss function is the 'scorekeeper' that tells the network exactly how wrong its current predictions are, while the optimizer is the 'strategist' that decides how to use that score to adjust the weights and do better on the next attempt — two distinct roles working together in every training loop.