In an InfiniBand-based high-performance computing (HPC) or AI fabric, which component is responsible for discovering the network topology, assigning Local Identifiers (LIDs) to nodes, calculating routing tables, and monitoring the fabric for changes?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Unlike Ethernet, which uses spanning tree and plug-and-play MAC learning, an InfiniBand network is centrally managed. It needs a "brain" to get up and running, and that brain is the Subnet Manager, or SM. When you power on your InfiniBand fabric, the SM wakes up and does a roll call. It discovers every switch and host, hands out Local Identifiers (LIDs)—which are like InfiniBand addresses—calculates the absolute best routing paths, and loads them onto the switches. If a cable gets yanked, the SM re-calculates the paths in real-time. Without a running SM, your InfiniBand network is just a bunch of expensive, quiet silicon. Remember: no SM, no traffic!
Full explanation below image
Full Explanation
An InfiniBand network operates as a centrally managed subnet, which distinguishes it architecturally from traditional, self-learning Ethernet networks. The central entity that administers an InfiniBand subnet is the Subnet Manager (SM). The SM can run either as a software service on a host node, embedded within an InfiniBand switch's firmware, or on a dedicated management appliance.
The Subnet Manager performs several critical phases of fabric management: 1. Topology Discovery: During initialization or topology changes, the SM sends Subnet Management Packets (SMPs) to discover all switches, channel adapters (HCAs), and links within the subnet. 2. LID Assignment: The SM assigns a unique 16-bit Local Identifier (LID) to each port in the subnet, which is used for routing packets within the subnet. 3. Routing Table Calculation: The SM calculates the optimal, loop-free routing paths between all endpoints and configures the forwarding tables of the switches. 4. Active Monitoring: The SM continuously polls the fabric to detect link failures, additions, or state changes, triggering a sweep and re-configuring the routing tables if any changes occur.
Let's look at the incorrect options: - Workload Managers like Slurm (Option A) operate at the application orchestration layer. They schedule jobs and decide which physical nodes run which parts of an AI training job, but they do not manage the physical or logical network layer. - Message Passing Interface / MPI (Option B) is a programming standard and library used for inter-process communication in parallel applications. It uses the network established by the SM but does not configure the hardware fabric. - Unified Fabric Manager / UFM (Option D) is an enterprise management platform that sits above the Subnet Manager to provide advanced monitoring, analytics, and orchestration, but standard fabric encryption and key management are not the defining primary duties of the basic Subnet Manager layer.