A multi-class image classifier's final layer produces raw, unbounded scores called logits. What does applying a softmax function to these logits accomplish?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal: softmax takes those raw, all-over-the-place logits and turns them into something you can actually reason about, a set of probabilities that all add up to 1. So instead of seeing a score of 7.2 for 'cat' and 4.1 for 'dog', you see 0.85 and 0.10. That's why the answer is converting logits into a probability distribution. It's not clipping values into a fixed range, that's more like what a tanh does. It's not pruning classes, every class still gets a probability, even a tiny one. And it's not a regularizer, it doesn't touch the weights at all, it's purely an output transformation applied at inference and during loss calculation.
Full Explanation
Softmax is an activation function typically applied to the final layer of a multi-class classification network. It takes a vector of raw, unbounded real-valued scores (logits) and transforms them into a vector of values that are all positive and sum to exactly one, effectively producing a probability distribution across the candidate classes. Mathematically, each output is the exponential of that class's logit divided by the sum of the exponentials of all logits, which guarantees both non-negativity and normalization. This makes softmax outputs interpretable as confidence scores and lets the network's output be directly compared against a one-hot encoded ground truth using cross-entropy loss.
The first distractor, clipping values to a fixed range like -1 to 1, describes something closer to the tanh activation function, not softmax; tanh squashes each input independently and does not produce values that sum to one, so it is not a probability distribution over classes. The second distractor, removing low-scoring classes, describes a completely different operation, more like top-k filtering or thresholding used in some sampling strategies (e.g., nucleus sampling), but softmax itself preserves every class in the output vector, simply assigning small probabilities to unlikely ones rather than discarding them. The third distractor, regularizing weights, confuses softmax with regularization techniques like L2 weight decay or dropout; softmax is an output activation with no effect on the weight update rule or model capacity.
A useful memory aid: softmax is a 'squash and share' operation, it squashes scores into a bounded range and shares out a total probability mass of 1.0 among the classes based on relative logit magnitude. This is why softmax is the standard choice for the output layer whenever a model needs to choose exactly one class from mutually exclusive categories, as opposed to sigmoid, which is used per-output for multi-label or binary problems where classes are not mutually exclusive.