A neuron computes a weighted sum of its inputs plus a bias term before applying an activation function. What role does the bias play?
Select an answer to reveal the explanation.
Short Explanation
Think about a straight line: y equals mx plus b. Without that 'b,' every line you could draw would have to pass through the origin — pretty limiting, right? A neuron's bias is that same 'b.' It lets the neuron shift its output up or down regardless of what the inputs are, so the model isn't stuck forcing every decision boundary through a fixed point. That's answer D. It's got nothing to do with scaling gradients during backprop — that's more a learning-rate or optimizer concern. It's not dropout, which randomly zeroes neurons, and it's not normalization, which rescales outputs into a range. Bias is simply the offset term that gives the neuron freedom to fit the data properly.
Full Explanation
In a neuron's computation, the output before activation is typically expressed as the weighted sum of inputs plus a bias term: z = (w · x) + b. The bias b acts as a constant offset that is independent of the input values, analogous to the intercept term in linear regression. Without a bias, the neuron's decision boundary (or the function it represents) would be constrained to pass through the origin, severely limiting the range of functions the network could learn. By including a learnable bias, each neuron gains the flexibility to shift its activation threshold, allowing the network to fit data that does not naturally center around zero and generally improving the model's capacity to represent real-world relationships.
The first distractor describes something closer to gradient scaling or learning-rate adjustment, mechanisms controlled by the optimizer (e.g., Adam's per-parameter scaling) rather than by the bias term itself; bias values are updated via gradients just like weights, but they do not themselves control gradient scaling for other parameters. The second distractor describes dropout, a regularization technique that randomly zeroes out a subset of neurons during training to prevent overfitting; this is a stochastic, training-time-only behavior entirely separate from what a bias term does. The third distractor describes normalization techniques such as batch normalization or layer normalization, which rescale activations to have controlled statistics (e.g., zero mean, unit variance) across a batch or layer; a bias term does not enforce any such rescaling, it simply adds a fixed learnable offset.
A simple memory aid: weights determine the slope or sensitivity of a neuron to each input, while the bias determines where that sloped response is centered — without it, every neuron's function would be needlessly anchored to the origin.