A city GPU InfiniBand fabric must start a large multi-rail job. What should operators verify first?

Before a city GPU InfiniBand fabric starts a large multi-rail job, operators should verify that InfiniBand adaptive routing is enabled so high-load AI runs can move flows when rails or spines congest. Multi-rail training amplifies collective traffic; relying on pre-optimized static routing tables still pins elephants to a subset of links and cannot react when congestion appears. Confirming AR readiness is a first-line civic bring-up check so the fabric can absorb the job’s burst without irreversible hot-spot pinning.

Relying on pre-optimized static routing tables fails by concept: it is the opposite of readiness for congestion-prone multi-rail AI and guarantees sticky imbalance because the paths cannot move when links heat up. Configuring UFM SHARP groups for background storage traffic fails because that tunes a specific offload feature rather than confirming the fabric can shift AI flows under congestion; it is not the pre-launch check operators need. Expecting subnet manager event logs to be forwarded to long-term audit storage fails because that is a post-run administrative task, not a pre-launch congestion control; it does not verify adaptive routing and does nothing to keep elephants off busy links when the job starts.

Exam caveat: enabling adaptive routing does not replace capacity checks or correct rail cabling; it is the congestion-response control to confirm before launch. Operational check: in SM/UFM, verify adaptive routing is on for the city GPU partition, note baseline rail utilization, start the multi-rail job, and confirm the fabric can rebalance rather than locking onto a single overloaded path.