You are training a very deep feedforward neural network using sigmoid activation functions. After several epochs, you observe that the weights of the layers closest to the input are barely changing, while the outer layers are training normally. What common neural network problem are you experiencing, and what is its primary cause?

The correct answer is vanishing gradients, where gradient values shrink exponentially as they backpropagate, making learning in early layers extremely slow. During backpropagation in a deep neural network, gradients are calculated using the chain rule, multiplying derivatives from the output layer back to the input layer. When using activation functions whose derivatives are less than 1, such as sigmoid, whose maximum derivative is 0.25, multiplying these small values repeatedly across many layers causes the gradient to decrease exponentially. Consequently, gradient updates for weights in layers closest to the input become virtually zero, halting their training and preventing the network from learning deep features. Pay close attention here, because this one bites people all the time, both in production and on the exam! The exploding gradients option is incorrect because exploding gradients occur when weights or derivatives are large, greater than 1, causing gradients to grow exponentially and lead to numerical instability, not slow learning due to small values or division by zero. The overfitting option is incorrect because overfitting is typically a symptom of a highly complex model memorizing the training data, rather than a simple model. The underfitting option is incorrect because underfitting is a symptom of a model that is too simple, or undertrained, to capture the underlying pattern of the data, not a highly complex model.