What is the main advantage of mini-batch gradient descent over full batch gradient descent when training on a large dataset?
Select an answer to reveal the explanation.
Short Explanation
Think of mini-batch as the sensible middle ground. Full batch gradient descent gives you a rock-solid, stable gradient estimate, but you have to crawl through the whole dataset just to take one step. Pure SGD (one example at a time) is fast and cheap per step, but wildly noisy. Mini-batch splits the difference: process a chunk of, say, 64 or 128 examples, get a reasonably stable gradient, and update way more often than full batch would let you. That's answer C — efficient on big data, balancing SGD's speed with batch's stability. It doesn't guarantee the global minimum, nobody does with non-convex neural network loss surfaces. It doesn't get rid of the learning rate — you still need one, in fact schedulers still tune it constantly. And you still absolutely want to shuffle your data between epochs to avoid learning spurious order effects.
Full Explanation
Mini-batch gradient descent divides the training set into small batches (commonly 32, 64, 128, or 256 samples) and updates the model's weights after processing each batch rather than after a single example (as in pure stochastic gradient descent) or the entire dataset (as in batch gradient descent). This gives two compounding benefits on large datasets: first, computing the gradient over a batch of examples produces a noticeably more stable estimate than a single-example gradient, smoothing out the erratic update path typical of pure SGD; second, because updates happen far more frequently than full-batch training, the model makes meaningful progress without waiting to traverse the whole dataset, and batches can be vectorized efficiently on GPU hardware.
The first distractor is incorrect because no gradient-descent variant guarantees convergence to a global minimum on the non-convex loss surfaces typical of deep neural networks; all variants can settle into local minima or saddle points, and mini-batch is no exception. The second distractor is incorrect because mini-batch gradient descent still requires a learning rate to scale each weight update — if anything, tuning the learning rate (often with schedules or adaptive optimizers) matters just as much with mini-batches as with any other variant. The third distractor is incorrect because shuffling the training data before each epoch remains a best practice with mini-batch training; without shuffling, batches would repeatedly contain the same groupings of examples in the same order, which can introduce bias into the learning process and hurt generalization.
In practice, mini-batch gradient descent (usually just called "SGD" in deep learning frameworks, even though it processes batches rather than single examples) is the default training strategy across nearly all modern deep learning because it delivers the best practical tradeoff between computational efficiency, memory usage, and gradient stability.