In terms of memory subsystem design, how do the hardware architectures of modern enterprise GPUs and general-purpose CPUs differ when handling data-heavy workloads?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Check this out: when it comes to memory, CPUs and GPUs have completely different philosophies. Let's use an analogy. Imagine you need to move a million gallons of water. A CPU memory subsystem is like a small pipe with a hyper-fast pump. The water travels fast (low latency), but there's only so much you can push through at once. A GPU, on the other hand, is like a massive river. The water moves slower (higher latency), but the sheer volume is astronomical (high bandwidth) because the path is incredibly wide. CPUs focus on getting a single piece of data to a core as fast as possible, using deep cache hierarchies (L1, L2, L3) and narrow memory buses (typically 64-bit per channel). GPUs don't care as much about individual latency; they care about feeding thousands of hungry cores simultaneously. That's why they use massive memory buses—sometimes 512-bit or even thousands of bits wide with HBM. So remember: CPU equals narrow and low-latency; GPU equals ultra-wide and high-bandwidth. Got it? Sweet.
Full explanation below image
Full Explanation
GPUs are designed for data-parallel processing, requiring massive amounts of data to be fed to thousands of cores simultaneously. To achieve this, GPU architectures utilize extremely wide memory interfaces. For example, traditional GDDR6 memory on GPUs can use bus widths of 256-bit to 512-bit or more, and stacked HBM (High Bandwidth Memory) architectures use silicon interposers to achieve bus widths of 1024-bit to 4096-bit per stack. This wide path allows GPUs to achieve massive bandwidth (up to multiple TB/s). In contrast, CPU memory subsystems are optimized for sequential, single-threaded execution, prioritizing low latency. CPUs use narrower memory interfaces (typically 64-bit per DDR channel, up to a few hundred bits total for multi-channel configurations) and depend heavily on multi-tiered cache hierarchies to reduce access latency. Let's analyze the incorrect options: - Option B is incorrect because CPUs do not consistently outperform GPUs in memory bandwidth; in fact, GPU memory bandwidth is typically one to two orders of magnitude higher than CPU memory bandwidth. Furthermore, enterprise CPUs do not generally use HBM as their primary memory. - Option C is incorrect because the opposite is true. CPUs rely heavily on massive cache hierarchies (L1, L2, L3 caches) to hide memory latency. GPUs have smaller caches relative to their core count and instead rely on large register files and hardware-based thread scheduling to hide memory latency (switching to active threads while waiting for memory fetches). - Option D is incorrect because CPUs and GPUs do not share the same memory bus width design; GPUs have significantly wider memory buses to achieve the high bandwidth necessary for parallel processing.