When you deploy a CUDA kernel to run on an NVIDIA GPU, the programming model must organize parallel execution units so they can scale across any number of GPU cores. What is the fundamental hierarchical structure used by CUDA to group individual execution threads together, and how are those groups further organized to execute the kernel?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of CUDA like a massive logistics operation. Individual threads are your workers, but managing them one by one is a headache. Instead, you bundle them into groups called thread blocks. Then, you load all those blocks into a grid to handle the entire job. The beauty of this is scalability: if you run the same grid on a bigger GPU with more Streaming Multiprocessors (SMs), it just runs more thread blocks at the same time. Got it? Sweet.
Full explanation below image
Full Explanation
The CUDA programming model uses a hierarchical execution model to abstract the GPU hardware and ensure transparent scalability across different generations of GPUs. At the lowest level, individual threads run the kernel code. These threads are grouped into thread blocks, which typically contain up to 1024 threads. All threads within a single thread block run on a single Streaming Multiprocessor (SM), sharing fast on-chip shared memory and synchronization barriers. These thread blocks are then organized into a grid. A grid represents the entire collection of thread blocks launched for a specific kernel. This abstraction is crucial for scalability: because thread blocks in a grid execute independently, the hardware scheduler can distribute them across any number of available SMs. A low-end GPU might execute thread blocks sequentially, while a high-end GPU with many more SMs will run those same blocks concurrently, accelerating execution without requiring recompilation of the code. - Options A, C, and D are incorrect because they refer to either OpenCL terminology (compute units), hardware-level scheduling units (warps), or incorrect structural hierarchies that do not define the CUDA software thread hierarchy.