Transit all-reduce traffic saturates a hot InfiniBand link while inference must stay responsive. What should operators apply?
Select an answer to reveal the explanation.
Short Explanation
Partition keys isolate subnets but don't prioritize inference on a hot link, and raising MTU just cuts packet overhead without giving inference the HOV lane. So you'll still need InfiniBand QoS classes to keep latency-sensitive civic inference responsive when all-reduce floods the link.
Full Explanation
When transit all-reduce traffic saturates a hot InfiniBand link, bulk collective bandwidth can starve latency-sensitive civic inference that must stay responsive for rider-facing or operations models. InfiniBand QoS classes are the correct application: map interactive inference to higher service levels (SL/VL) with preferential arbitration so those queues keep forwarding preference while heavy all-reduce remains in a lower or best-effort class on the same physical fabric. That separation lets municipal transit operators protect SLA-bound inference without shutting training down.
Assigning all-reduce and inference to matching QoS levels fails because equal service levels remove prioritization; both flows then contend fairly and all-reduce still dominates due to its bulk bandwidth. Configuring InfiniBand partition keys to isolate traffic into subnets fails because it doesn't provide QoS prioritization on the shared hot link; all-reduce can still saturate the link within its partition, starving latency-sensitive inference. Raising InfiniBand MTU for all-reduce fails because it only reduces packet rate without providing priority to inference; all-reduce can still monopolize the link's bandwidth, leaving latency-sensitive inference packets to wait.
Exam caveat: QoS protects relative priority on shared links; it does not create free bandwidth if the fabric is chronically undersized, and adaptive routing may still be needed to spread hot spots. Operational check: verify SL/VL mappings for transit inference versus all-reduce in the SM/UFM QoS configuration, generate mixed load, and confirm inference latency stays within SLA while all-reduce continues at lower priority on the hot link.