When building infrastructure to train deep learning models at scale, engineers almost universally choose Graphics Processing Units (GPUs) over standard Central Processing Units (CPUs). What is the primary hardware advantage that makes GPUs so superior for this task?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Imagine your boss walks in and asks you to solve two million simple addition problems by Friday. A CPU is like a genius mathematician who is incredibly fast but can only work on one problem at a time. A GPU, however, is like an army of thousands of elementary school kids working on a chalkboard simultaneously. Because training neural networks is really just doing billions of simple matrix multiplications over and over, the GPU's parallel architecture leaves the CPU in the dust. It doesn't save memory or power, and it doesn't change your code—it just gives you raw, parallelized speed. That's why Option A is the right answer.
Full explanation below image
Full Explanation
Deep learning models, particularly large convolutional neural networks or transformers, are mathematically dominated by linear algebra operations. Forward propagation and backpropagation require performing billions of matrix multiplications, vector additions, and tensor transformations.
A Central Processing Unit (CPU) is built for general-purpose sequential processing. It contains a small number of cores optimized for single-threaded performance and low-latency execution of complex logic branches. Conversely, a Graphics Processing Unit (GPU) is designed with a highly parallel architecture consisting of thousands of smaller, simpler cores. These cores are optimized to perform identical mathematical operations on multiple data points simultaneously (Single Instruction, Multiple Data, or SIMD). Consequently, when a training job is offloaded to a GPU, the massive matrices representing model weights and training activations can be multiplied in parallel rather than sequentially, reducing training times from weeks to hours.
Here is why the other options are distractors: - Option B is incorrect because GPUs are highly power-hungry components that draw significant wattage (often 300W to 700W+ per card) and generate substantial heat, thereby increasing rather than reducing energy consumption. - Option C is incorrect because hardware accelerators have no impact on the complexity or architecture of the neural network model; model architecture is determined purely by the design choices in software. - Option D is incorrect because deep learning on GPUs actually requires significant memory allocations (VRAM or HBM) to hold model weights and gradients, and does not compress or reduce overall memory requirements.
For the exam, remember that the core benefit of a GPU in deep learning is the acceleration of linear algebra and matrix operations via parallel computation.