Your team is transitioning a deep learning model training workload from high-performance multi-core CPUs to NVIDIA GPUs. The model processes massive datasets and performs complex matrix multiplications. Which fundamental architectural feature makes GPUs more suitable than CPUs for this task?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of it like this: a high-end CPU is like a Ferrari. It has a super fast engine (high clock speed) and is designed to get a couple of passengers from point A to point B incredibly fast. But what if you need to move a thousand people at the same time? That Ferrari is useless. You need a thousand mini-buses running in parallel. That's a GPU. While a CPU has a few highly optimized, complex cores, a GPU has thousands of smaller, simpler cores. Deep learning is just massive matrix multiplication—millions of simple math problems that can all be done at the exact same time. The GPU's massive parallel architecture lets it tackle all these math problems at once, leaving the CPU in the dust. And don't get fooled on the exam: CPUs actually have much larger caches and higher clock speeds than GPUs!
Full explanation below image
Full Explanation
The fundamental difference between CPU and GPU architectures lies in their design philosophies. A Central Processing Unit (CPU) is designed for general-purpose sequential computing. It features a few powerful cores optimized to execute a single thread of instructions as quickly as possible. To achieve this, CPU cores allocate significant die area to large cache memories (L1, L2, L3) to reduce latency and complex control logic like branch prediction and out-of-order execution. In contrast, a Graphics Processing Unit (GPU) is designed for high-throughput parallel workloads. Deep learning, especially training neural networks, consists primarily of matrix multiplications and vector additions. These workloads can be broken down into millions of independent arithmetic operations that can run simultaneously (Data Parallelism). To exploit this, a GPU allocates most of its hardware die area to Arithmetic Logic Units (ALUs)—the actual calculation engines—resulting in thousands of cores on a single chip. While individual GPU cores are simpler and run at lower clock speeds than CPU cores, their sheer number allows them to process massive batches of data concurrently. Let's analyze the incorrect options: - Large cache memory (Option A) is a hallmark of CPUs, which use it to minimize memory latency; GPUs have relatively small caches per core, relying instead on high-bandwidth memory (HBM) and latency hiding through massive multithreading. - High core clock speed (Option C) is also a CPU characteristic; CPU cores typically run at 3.0 GHz to 5.0 GHz, while GPU cores run at much lower speeds (typically 1.2 GHz to 2.0 GHz). - Out-of-order execution (Option D) is a latency-minimization technique used by CPUs to optimize sequential execution, whereas GPUs use simple in-order execution pipelines because their workloads are dominated by uniform parallel tasks. In summary, while a CPU is optimized for low latency in sequential operations, a GPU is designed for high throughput in parallel operations. The transition to GPUs allows deep learning training to scale by exploiting this parallelism, turning what would be a sequential bottle-neck on a CPU into a highly distributed, simultaneous computation. Therefore, the GPU's massive parallel design is the primary factor driving deep learning acceleration.