In multi-node GPU clusters, transferring data between GPUs on different servers can introduce significant latency. Which NVIDIA technology allows a Remote Direct Memory Access (RDMA) network interface card (NIC) to read or write data directly from or to GPU memory across the network fabric, completely bypassing the host CPU and system RAM?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Pay close attention here because this concept shows up all the time on the exam! In a standard multi-node setup, if GPU 1 on Server A wants to send data to GPU 2 on Server B, the data has to make a lot of stops. It travels from GPU 1, over the PCIe bus, into the host CPU's system RAM, then to the local network card, across the wire, into Server B's host RAM, and finally back down to GPU 2. That is a massive traffic jam! GPUDirect RDMA solves this by letting the network interface card (NIC) talk directly to the GPU's memory. It pulls the data straight out of GPU 1's memory and pushes it directly into GPU 2's memory across the network. No host CPU interrupts, no host RAM overhead. It's clean, fast, and exactly how we keep latency low in distributed AI workloads.
Full explanation below image
Full Explanation
Distributed GPU computing depends on rapid communication between GPU nodes. When training large language models or running large-scale physical simulations, GPUs on separate servers must exchange massive amounts of data. Without optimization, this data transfer requires copying the data to host system memory (CPU RAM) before it can be transmitted via the network interface card (NIC).
NVIDIA's GPUDirect RDMA is the specific technology designed to eliminate this bottleneck. It leverages PCIe peer-to-peer (P2P) communication to create a direct path between the GPU's memory and an RDMA-capable network adapter (such as a Mellanox NIC). This allows the NIC to directly read or write GPU memory pages across the network, completely bypassing the host CPU's memory controllers and system RAM. This direct path slashes data transfer latency, reduces overhead on the host CPU, and increases overall network throughput.
Let's look at the other options: - GPUDirect Storage (GDS) establishes a direct path between local or remote storage (like NVMe drives) and GPU memory, bypassing the CPU bounce buffer. While it is part of the GPUDirect family, its target is storage I/O rather than node-to-node networking. - CUDA Unified Virtual Addressing (UVA) creates a single virtual memory address space across the host and the GPU devices, allowing pointers to be used seamlessly, but it does not manage network-level RDMA transfers. - NVLink is a physical, high-bandwidth interconnect designed for GPU-to-GPU communication within a single node or chassis, rather than across external network fabrics.