What is the purpose of batch gradient descent when training a model?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Gradient descent walks downhill on the loss surface by stepping opposite the gradient. Batch gradient descent means each step looks at the whole training set before you move the weights. Accurate direction, but can be slow and memory-heavy on huge data. SGD glances at one example — noisy but fast. Mini-batch is the usual compromise. Learning rate? That's the step size knob, not what batch means. Exam trap: mixing batch, mini-batch, and stochastic. If the question says entire dataset per update, that's batch. I remember when teams said batch and meant mini-batch in slang — on the test, read carefully. Full dataset per update equals batch GD.
Full explanation below image
Full Explanation
Gradient descent optimizes model parameters by iteratively moving them in the direction that reduces a differentiable loss. The three classic variants differ by how much data contributes to each gradient estimate. Batch gradient descent, also called full-batch, computes the gradient of the loss summed or averaged over every training example, then performs a single parameter update per pass over that full gradient. The update direction is stable and exact for the empirical risk on the training set, which can yield smooth convergence behavior, but each step is expensive when the dataset is large, and the method can be less able to escape certain shallow local issues compared with noisier estimators.
Stochastic gradient descent updates after each individual example or a stream of them, introducing high variance in the gradient estimate but cheap updates and often better scalability. Mini-batch gradient descent computes gradients on small random subsets and dominates modern deep learning practice because it balances noise, parallelism on GPUs, and throughput. None of these definitions is identical to finding the optimal learning rate; the learning rate and schedules or adaptive methods such as Adam control step size and are chosen separately, though they interact with batch size.
For exam questions, map wording precisely: entire training set per update means batch; one example means stochastic; random subset means mini-batch. Do not confuse batch as a casual synonym for any training step with the technical full-batch meaning. Also separate optimization algorithm choice from model evaluation and from regularization. Remember that batch size is a key practical hyperparameter even when using mini-batches because it affects gradient noise, generalization, and hardware utilization—but the defining purpose of batch gradient descent remains full-dataset gradient updates of the weights.
A compact comparison helps under exam pressure: batch is accurate but costly per step; stochastic is cheap and noisy; mini-batch is the industry default compromise. If an option talks only about choosing a learning rate, it is describing a hyperparameter decision, not the definition of batch gradient descent. Choose the option that mentions updating weights after processing the entire training dataset.