You're scaling distributed training across multiple nodes, and you're trying to minimize the latency of inter-GPU memory access when gradients need to be exchanged between GPUs on different nodes. Which NVIDIA technology is optimized for exactly this scenario?
Select an answer to reveal the explanation.
Short Explanation
GPUDirect RDMA optimizes remote GPU-to-GPU communication across the network. Reduced latency for inter-GPU memory access across networked systems is the exact use case. The other options describe different capabilities (local memory sharing, GPU-CPU transfers, power efficiency).
Full Explanation
GPUDirect RDMA enables a GPU on Node A to directly read from or write to a GPU's memory on Node B, without intermediate buffering on CPUs or host memory. This is critical in distributed training. When GPUs synchronize gradients or parameters, they need to exchange large volumes of data. Without RDMA, the data flows: GPU A → CPU A → network → CPU B → GPU B. That's latency overhead and CPU contention. GPUDirect RDMA flips it: GPU A → network → GPU B directly. The latency plummets, and the CPU stays out of the picture. This is especially valuable in large-scale distributed training (8+ GPUs across multiple nodes) where synchronization overhead becomes a significant part of training time. GPUDirect has several flavors: GPUDirect P2P (GPU-to-GPU within a node via NVLink), GPUDirect Storage (GPU-to-storage), and GPUDirect RDMA (GPU-to-GPU across network). The question specifically asks about inter-GPU memory access across networked systems, which is RDMA.