In one sentence, what is gradient descent fundamentally trying to accomplish when training a model?
Select an answer to reveal the explanation.
Short Explanation
Gradient descent is basically hiking downhill in the fog: you can't see the whole landscape, but you can feel which way slopes down the steepest right where you're standing, so you take a step that way, check again, and repeat. Do that over and over and you work your way toward a minimum of the loss function, and that's the whole point of the algorithm.
Full Explanation
The statement that says find a minimum of the loss function by repeatedly moving the weights in the direction of steepest descent is correct because gradient descent is an iterative optimization algorithm whose fundamental goal is to find a minimum of a loss function by adjusting the model's parameters, including weights and biases, step by step. At each iteration, the algorithm computes the gradient of the loss with respect to the current parameters. The gradient is a vector that points in the direction of steepest increase of the loss, so gradient descent updates the parameters by moving a small step in the opposite direction, the direction of steepest decrease, scaled by the learning rate. Repeating this process causes the loss to decrease over successive iterations until it converges to a minimum, ideally the global minimum, though in practice often a good local minimum or flat region, especially in the high-dimensional, non-convex loss surfaces typical of deep neural networks. The statement that says maximize the loss function so the model becomes more confident in its predictions is incorrect because it inverts the objective: gradient descent minimizes the loss, it does not maximize it. Maximizing a loss function would make the model's predictions worse, not better. The statement that says randomly search the weight space until validation accuracy stops improving describes an undirected random search strategy. Gradient descent is specifically valuable because it uses the gradient, calculus rather than randomness, to choose an informed direction for each update, which is dramatically more efficient than blind search, especially in high-dimensional parameter spaces with millions of weights. The statement that says compute the exact global optimum of the loss function in a single closed-form step describes a closed-form analytical solution, such as the normal equation used to solve ordinary least squares linear regression directly in one step. Gradient descent is an iterative numerical method used precisely in situations where a closed-form solution is unavailable, computationally infeasible, such as due to matrix inversion cost at scale, or does not exist because the loss surface is nonlinear and non-convex, as with deep neural networks. Memory aid: 'gradient points uphill, so you walk the negative gradient downhill, one step at a time, until you reach the bottom.' This simple iterative mechanism, combined with the chain rule for computing gradients through many layers, backpropagation, is the engine that trains virtually all modern neural networks.