In an NVIDIA DGX H100 system, which proprietary high-bandwidth interconnect is utilized for direct GPU-to-GPU communication, bypassing the traditional PCIe bus?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of NVLink like a high-speed superhighway connecting the GPUs inside the chassis. If the GPUs had to talk to each other over the standard PCIe bus, it would be like routing highway traffic through a narrow city street—it becomes a massive bottleneck! NVIDIA solved this by building NVLink. It's a dedicated, high-bandwidth interface that connects the GPUs directly to each other. In the DGX H100, the GPUs use NVLink switches to share data at mind-blowing speeds. Now, don't confuse this with external networking. If your DGX needs to talk to other DGX systems in the cluster, that's when you use InfiniBand. For general management and storage, you'll use Ethernet. But for GPU-to-GPU communication inside that box? NVLink is the king.
Full explanation below image
Full Explanation
The NVIDIA DGX H100 is a high-performance AI system containing eight NVIDIA H100 Tensor Core GPUs. To train large AI models efficiently, these GPUs must constantly exchange parameters, gradients, and synchronization signals. Standard system buses, such as PCIe, do not provide sufficient bandwidth and introduce too much latency for these workloads.
To solve this, NVIDIA utilizes NVLink, a proprietary, bidirectional, high-bandwidth interconnect. Inside the DGX H100 system, NVLink connects all eight GPUs in a fully meshed topology via NVLink Switches. This allows direct GPU-to-GPU communication with bandwidths far exceeding standard PCIe slots (up to 900 GB/s per GPU in the H100 generation), enabling the GPUs to function as a single, massive accelerator.
Let’s review the other options: - InfiniBand is the high-throughput, low-latency clustering network used for node-to-node (external) communication between different DGX systems. It is not the internal GPU-to-GPU interconnect. - Ethernet is typically used for cluster management, user access, and slower storage connections, rather than high-performance inter-GPU communication. - PCIe (PCI Express) is the standard system bus used to connect the host CPU to GPUs and other peripherals. While the GPUs are connected to the host via PCIe, NVLink is the primary path for inter-GPU traffic because of its vastly superior bandwidth.