When designing a neural network to perform multi-class classification (where an input must be assigned to one of several mutually exclusive categories), which activation function should be applied to the final output layer?
Select an answer to reveal the explanation.
![A diagram showing the final layer of a neural network processing 3 logits (e.g., [2.0, 1.0, 0.1]) passing through a Softmax block, which outputs a probability distribution vector (e.g., [0.7, 0.2, 0.1]) summing to 1.0, labeled with class names (e.g., Dog, Cat, Bird).](/media/blockchain-council/caid/CAID-e2-q097.webp)
Short Explanation
When you're doing multi-class classification, you want the output layer to turn raw scores into probabilities that add up to 1. Softmax does that, so you can pick the most likely mutually exclusive category. Sigmoid, ReLU, and tanh don't give you that clean probability distribution.
Full Explanation
In multi-class classification, the goal is to classify an input into one of several mutually exclusive classes. The output layer of the neural network typically produces raw, unnormalized real-valued scores, known as logits, for each class. To interpret these logits as probabilities, you apply the Softmax activation function. Softmax exponentiates each output score and normalizes it by dividing by the sum of all exponentiated scores. This ensures two properties required for a probability distribution: every output probability lies in the range from 0 to 1, and the sum of all output probabilities across all classes is exactly equal to 1.0.
The Rectified Linear Unit output layer is inappropriate because it outputs the maximum of zero and the input, is unbounded on the positive side, and therefore does not produce probabilities that can be directly used to compute multi-class cross-entropy loss; it is primarily used in hidden layers. The Sigmoid output layer is inappropriate for multi-class classification because it maps each output independently to the range from 0 to 1, so the outputs do not sum to 1; it is the correct choice for binary classification or multi-label classification, where an input can belong to multiple categories simultaneously. The Hyperbolic Tangent output layer is inappropriate because it maps inputs to the range from -1 to 1, and probabilities cannot be negative.