In one sentence, what is gradient descent fundamentally trying to accomplish when training a model?
Select an answer to reveal the explanation.
Short Explanation
Gradient descent is basically hiking downhill in the fog: you can't see the whole landscape, but you can feel which direction slopes down the steepest right where you're standing, so you take a step that way, check again, and repeat. Do that over and over and you work your way toward a minimum of the loss function — that's answer C, and it's the whole point of the algorithm. Option A has it backwards; you want to minimize loss, not maximize it, since lower loss means better predictions. Option B describes random search, which is wildly inefficient and ignores the gradient information you actually have available. And option D describes a closed-form solution, like the normal equation in plain linear regression — gradient descent is the iterative alternative you reach for precisely when no such single-step formula exists or is practical for a large, nonlinear model.
Full Explanation
Gradient descent is an iterative optimization algorithm whose fundamental goal is to find a minimum of a loss function by adjusting the model's parameters (weights and biases) step by step. At each iteration, the algorithm computes the gradient of the loss with respect to the current parameters. The gradient is a vector that points in the direction of steepest increase of the loss, so gradient descent updates the parameters by moving a small step in the opposite direction — the direction of steepest decrease — scaled by the learning rate. Repeating this process causes the loss to decrease over successive iterations until it converges to a minimum (ideally the global minimum, though in practice often a good local minimum or flat region, especially in the high-dimensional, non-convex loss surfaces typical of deep neural networks).
The first distractor is incorrect because it inverts the objective: gradient descent minimizes the loss, it does not maximize it. Maximizing a loss function would make the model's predictions worse, not better.
The second distractor describes an undirected random search strategy. Gradient descent is specifically valuable because it uses the gradient — calculus, not randomness — to choose an informed direction for each update, which is dramatically more efficient than blind search, especially in high-dimensional parameter spaces with millions of weights.
The fourth distractor describes a closed-form (analytical) solution, such as the normal equation used to solve ordinary least squares linear regression directly in one step. Gradient descent is an iterative numerical method used precisely in situations where a closed-form solution is unavailable, computationally infeasible (e.g., due to matrix inversion cost at scale), or does not exist because the loss surface is nonlinear and non-convex, as with deep neural networks.
Memory aid: 'gradient points uphill, so you walk the negative gradient downhill, one step at a time, until you reach the bottom.' This simple iterative mechanism, combined with the chain rule for computing gradients through many layers (backpropagation), is the engine that trains virtually all modern neural networks.