From a software development perspective, how does the CUDA programming model hierarchically organize parallel execution paths to map workloads onto NVIDIA GPU hardware?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Let's dive into CUDA programming. GPUs have thousands of cores, but how do you actually write code to run on them? You don't manually map tasks to physical transistors. Instead, CUDA gives you a logical thread hierarchy. Think of it like this: your code is run by individual threads. You group those threads into thread blocks, and then group those blocks into a grid. When you launch a CUDA kernel, you're launching a grid. The GPU hardware takes care of mapping those blocks onto its physical Streaming Multiprocessors (SMs). Don't get tripped up here: SMs and Tensor Cores are physical hardware parts, not the software execution hierarchy. Trust me, the exam wants you to know this logical-to-physical difference!
Full explanation below image
Full Explanation
The CUDA programming model abstracts NVIDIA GPU hardware, enabling developers to write parallel code using a logical execution hierarchy: - Thread: The basic unit of execution. Each thread executes a copy of the kernel code. - Thread Block (or Block): A grouping of threads that execute on the same Streaming Multiprocessor (SM) at the same time. Threads within a block can communicate with each other via fast shared memory and synchronize their execution. - Grid: A collection of thread blocks that execute the same kernel. The grid represents the entire parallel execution space launched by the host. This hierarchy (Option D) is a logical software construct. During execution, the GPU's hardware scheduler assigns thread blocks to physical Streaming Multiprocessors (SMs) (Option A). The SMs contain physical CUDA cores and Tensor Cores (Option C), which perform the actual computations. Sequential pipelines (Option B) are the opposite of CUDA's massive parallel design. This logical abstraction allows the same CUDA program to scale transparently across different GPUs. A program written with many blocks will run on a small GPU (with few SMs) by scheduling blocks sequentially, and on a larger GPU (with many SMs) by scheduling blocks in parallel. The developer does not need to know the physical number of SMs at compile time. SMs represent hardware execution resources, while Tensor Cores are specialized hardware units for accelerating matrix multiplication.