In deep learning training loops, what is a batch?
Select an answer to reveal the explanation.
Short Explanation
Here's the deal: you rarely shove the entire dataset through the network for every update—that can be huge and slow. You grab a batch: a chunk of examples, run them together, average the gradient, step the weights. Rinse and repeat. It's not "how wide is my hidden layer." It's not forced to be one lonely example (though batch size 1 is allowed). And it's not automatically "the whole dataset"—that's full-batch GD, a special case. Think of a dishwasher: you wash a load, not one spoon and not the whole kitchen at once every cycle. Exam land it: batch = subset per gradient update.
Full Explanation
Stochastic gradient-based optimization almost always operates on mini-batches. A batch is a group of training examples sampled (often shuffled each epoch) that are processed together in one iteration: the model computes predictions for all items in the batch, aggregates the loss, backpropagates to obtain a gradient estimate, and the optimizer updates parameters once. Batch size balances gradient noise, hardware utilization, and generalization behavior. Larger batches yield stabler gradient estimates and better accelerator occupancy but may need learning-rate retuning (linear scaling rules are a common heuristic); smaller batches inject beneficial noise that can help escape sharp minima but underuse GPUs if chosen too tiny relative to device memory.
The width of a hidden layer is independent of batching; you can change the number of neurons without changing how training examples are sliced into groups. Defining a batch as always a single example confuses terminology: online learning uses batch size 1, but the word "batch" still refers to the set used for the update—and practitioners routinely choose 32, 128, 1024, or more depending on model and hardware. Calling a batch the entire training dataset describes full-batch gradient descent, which is uncommon for large deep learning problems because of memory limits and wall-clock cost; in everyday DL language, batches are subsets, and one full pass over all subsets constitutes an epoch.
Related terms include mini-batch SGD, gradient accumulation (simulating larger effective batches when memory is tight by summing gradients over micro-batches before stepping), and data loaders that collate samples into tensors whose leading dimension is batch size N. Drop-last and padding strategies handle uneven final batches. Memory aid: "Batch = the examples in this update step." Exam answers should emphasize a subset used for one gradient and weight update, not hidden-layer size, not a mandatory singleton, and not the whole corpus by default.