When analyzing the hardware architectures of modern processors for data-intensive AI workloads, which statement accurately describes the fundamental differences in memory bandwidth and cache design between CPUs and GPUs?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of it like this: a CPU is like a high-end sports car. It's designed to fetch a small package from the store incredibly fast—that's low latency. A GPU is like a massive 100-lane highway filled with delivery trucks. It takes a fraction of a second longer to start moving (higher latency), but it transports a mountain of cargo all at once—that's high bandwidth. To feed their thousands of parallel cores, GPUs use wide memory buses (like 256-bit, 512-bit, or even wider with HBM) and specialized memory types (GDDR/HBM) to achieve transfer speeds of 400 to well over 1000 GB/s. CPUs, on the other hand, focus on serial execution speed and low-latency cache access, running on standard system DDR memory that tops out around 50 to 200 GB/s. Don't mix up latency and bandwidth on your exam!
Full explanation below image
Full Explanation
The architectural variance between CPUs and GPUs is driven by their differing processing paradigms. CPUs are latency-optimized processors designed for serial execution, while GPUs are throughput-optimized processors designed for massive parallel execution.
GPUs require massive data throughput to feed thousands of arithmetic logic units (ALUs) simultaneously without stalling. To achieve this, GPU architectures prioritize memory bandwidth. They use wide memory interfaces (often 256-bit or 512-bit for GDDR, and up to 1024-bit or wider per stack for High Bandwidth Memory, or HBM) and high-speed memory architectures. This design allows modern GPUs to routinely achieve memory bandwidths ranging from 400 GB/s to well over 1000 GB/s (and exceeding 3 TB/s in architectures like H100).
In contrast, CPUs are designed to execute single-threaded tasks as quickly as possible. They rely on large, low-latency cache hierarchies (L1, L2, and L3 caches) to keep data close to the execution cores, minimizing access latency. Because they connect to system RAM via narrower memory buses (typically dual-channel or quad-channel DDR configurations), their raw memory bandwidth is much lower, generally falling between 50 and 200 GB/s.
Let's review the incorrect options: - A is incorrect because caching decreases latency for frequently accessed data, but it does not increase the physical, raw bandwidth of the memory bus. GPUs have far higher raw bandwidth than CPUs. - B is incorrect because GPUs do not sacrifice memory bandwidth; they actively maximize it to prevent starving their compute cores. - D is incorrect because their bandwidth profiles are completely different, and the performance gap is rooted in physical hardware design (bus width, memory channels, memory type), not software drivers.