In supervised machine learning, what is a loss function?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: training is basically "guess, measure how wrong you were, nudge weights, repeat." That measure of wrongness? The loss function. Mean squared error for regression, cross-entropy for classification—different flavors, same idea. Think of a GPS yelling how many meters off-route you are; gradient descent steers using that signal. Regularization can ride along in the total objective to fight overfitting, but loss isn't "the overfitting button." It doesn't draw your architecture, and it doesn't pick the learning rate. Exam trap: mixing loss with regularization, LR schedules, or pretty diagrams. Land it: loss = how bad are the predictions right now? Optimizers live to drive that number down. Practice naming one loss per problem type and you're golden.
Full explanation below image
Full Explanation
Supervised learning fits parameters θ so that predictions f(x; θ) match labels y. A loss function L(y, f(x; θ)) assigns a non-negative (or otherwise ordered) penalty to mismatches. Aggregating loss over a batch or full dataset yields the training objective. Differentiation of that objective with respect to parameters produces gradients used by optimizers (SGD, Adam, and variants). Common examples: mean squared error and MAE for regression; binary and categorical cross-entropy for classification; specialized losses for ranking, detection, or imbalanced data.
The correct definition emphasizes measuring predictive discrepancy to drive learning. Regularization (L2 weight decay, dropout, early stopping) addresses overfitting; weight decay often appears as an added term in the total objective, but calling the loss "a function used to prevent overfitting" confuses the error measure with a capacity-control technique. Architecture visualization is tooling, not an objective. Learning rate controls optimizer step size and may be scheduled, but it is not defined by the loss function itself—though loss landscape shape interacts with which rates work well.
Principles: choose a loss aligned with the task and evaluation metric when possible; monitor train and validation loss for over/underfitting; remember that "metrics" (accuracy, F1) evaluate quality while "loss" is what training differentiates. Underlying principle: the loss is the differentiable training objective that measures prediction error, not a synonym for regularization, diagrams, or learning-rate schedules. Best practice selects a loss aligned with the task, monitors train and validation loss jointly, and separates metrics used for reporting from the objective used for optimization. Memory aid: loss is the scoreboard the optimizer reads every step.