What is an advantage of batch gradient descent over stochastic gradient descent (SGD)?
Select an answer to reveal the explanation.
Short Explanation
Batch gradient descent's whole appeal is stability. Because it computes the gradient using the entire dataset every single time before taking a step, that gradient estimate is about as accurate and noise-free as it gets — no single weird example is going to yank the update in a bad direction. The result is a smooth, steady, predictable path down toward the minimum, without the jittery zigzagging you see with single-example SGD. That's answer B. It's not about using less memory — that's actually backwards, batch GD needs to process the whole dataset at once, which is memory-heavy, while SGD needs just one example at a time and is far lighter. It doesn't let you update before collecting data — you obviously need actual data to compute any gradient at all. And it still absolutely needs a loss function; that's what defines the very error the gradient is computed from, no optimizer can skip that.
Full Explanation
Batch gradient descent computes the gradient of the loss function using the full training dataset before performing each parameter update. Because this gradient is effectively an average taken over every training example, individual outliers or noisy examples have much less influence on any single update than they would in stochastic gradient descent, which computes its gradient from just one example at a time. This averaging effect produces a smooth, low-variance descent path toward a minimum, in contrast to the jagged, high-variance trajectory typical of pure SGD, making batch gradient descent's convergence behavior more predictable and stable, particularly useful for analyzing convergence theoretically or on smaller datasets where computing the full-batch gradient is not prohibitively expensive.
The first distractor is incorrect and actually inverts the real tradeoff: batch gradient descent requires processing the entire dataset to compute a single gradient, which is memory-intensive, especially for large datasets, whereas SGD needs only one example in memory at a time, making SGD far lighter on memory per update, not heavier. The second distractor is incorrect because any gradient-based optimization method, including batch gradient descent, requires actual labeled training data to compute a gradient at all — there is no mechanism for updating weights before data exists, since the gradient is derived directly from comparing predictions against real target values. The third distractor is incorrect because a loss function is fundamental to any gradient descent variant: it defines exactly what quantity is being minimized, and without it there would be nothing to differentiate or compute a gradient from, regardless of whether the gradient is computed over one example, a mini-batch, or the entire dataset.
The key tradeoff with batch gradient descent is that its stability comes at the cost of speed and scalability: because a full pass over the dataset is required for every single update, training becomes prohibitively slow and memory-intensive on very large datasets, which is why mini-batch gradient descent — offering a middle ground between the smooth stability of full-batch updates and the frequent, lightweight updates of pure SGD — is the standard choice in most modern deep learning workflows.