During the training loop, after the network makes a prediction for a batch of examples, something has to quantify how wrong that prediction was so the optimizer knows which direction to adjust the weights. What is the name of that component, and what role does it play?
Select an answer to reveal the explanation.
Short Explanation and Infographic
That's the loss function's job. It takes the network's prediction and the actual correct answer and boils the difference down into a single number — a scalar — that says exactly how wrong the model was on that batch. Everything downstream depends on that number: backpropagation computes gradients of the loss with respect to every weight, and the optimizer uses those gradients to nudge weights in the direction that should shrink the loss next time. It's easy to mix up loss function and optimizer, but they're not the same thing — the loss function measures the error, while the optimizer is the algorithm (like SGD or Adam) that decides how to use that error signal to update weights. The activation function is about introducing nonlinearity inside the network's layers, and the tokenizer is strictly an NLP preprocessing step turning text into numbers — neither one measures prediction error.
Full explanation below image
Full Explanation
The correct answer is the loss function (also called the cost function or objective function), whose role is to take the model's predictions and the corresponding ground-truth labels for a batch and compute a single scalar value representing how far off those predictions are. Common examples include mean squared error for regression tasks and cross-entropy loss for classification tasks. This scalar is the quantity that the entire training process is built around: backpropagation computes the gradient of the loss with respect to every weight in the network via the chain rule, and those gradients tell the optimizer exactly which direction and how strongly to adjust each weight in order to reduce the loss on future predictions. Without a loss function, there would be no well-defined notion of 'error' for the network to reduce, and the whole gradient-based training loop would have nothing to optimize against. The first distractor, the activation function, serves a completely different purpose inside the network's architecture: functions like ReLU, sigmoid, or tanh introduce nonlinearity into each layer's output, which is what allows a neural network to model complex, non-linear relationships between inputs and outputs — activation functions operate at every layer during the forward pass and have no role in quantifying overall prediction error at the end of that pass. The second distractor, the tokenizer, is a preprocessing component specific to NLP pipelines that converts raw text into numeric token IDs before the data ever enters the network; it operates before training even reaches the model's computation and has nothing to do with measuring how wrong a prediction is. The third distractor, the optimizer, is a closely related but distinct component from the loss function: the optimizer (examples include stochastic gradient descent, Adam, and RMSprop) is the algorithm that uses the gradients derived from the loss function to actually update the weights — it consumes the output of the loss function's gradient computation but does not itself measure the distance between predicted and true values; that measurement is precisely the loss function's job. A clean way to keep these straight: the loss function tells you how wrong you are, backpropagation tells you which way each weight contributed to that wrongness, and the optimizer decides how big a step to take to fix it.