Why must we apply non-linear activation functions (like ReLU or Sigmoid) between the layers of a deep neural network rather than just stacking linear layers?

In deep learning, the capability of neural networks to solve highly complex tasks stems from their architectural depth and non-linear properties. Introducing non-linearity (Option C) is the fundamental purpose of activation functions in a neural network. A linear transformation is represented as f(x) = wx + b. If we stack multiple layers that only perform linear transformations, the composition of these layers is mathematically equivalent to a single linear transformation: y = W_3(W_2(W_1 x + b_1) + b_2) + b_3 = W_effective x + b_effective. Without a non-linear activation function, a neural network with a hundred layers would have no more representational power than a simple linear regression model. It would be incapable of learning complex patterns, curves, or decision boundaries (such as the XOR gate problem or image recognition features). Non-linear functions like ReLU (Rectified Linear Unit), Sigmoid, or Tanh allow the network to approximate virtually any continuous function (the Universal Approximation Theorem). Weight initialization (Option A) is handled by specific initialization algorithms (e.g., Xavier or He initialization) before training begins, not by the activation function. Loss calculation (Option B) is performed by a loss function (like Cross-Entropy or Mean Squared Error) at the output layer to evaluate prediction error, not by the activation functions within the hidden layers. Overfitting (Option D) is actually more likely in highly complex non-linear models. Regularization techniques (like dropout or L2 regularization) are used to mitigate overfitting, not activation functions. Therefore, activation functions are required to introduce non-linearity, enabling deep neural networks to model complex relationships.