A public-health InfiniBand fabric sees static paths congest during model training. What should operators configure?

In a public-health InfiniBand fabric, static routes create fixed forwarding trees that keep training collectives on the same spine or leaf paths even as those paths saturate. InfiniBand adaptive routing is the correct control because the subnet manager and switches can reselect alternate LID paths when congestion or credit starvation appears, so model-training flows detour instead of queuing endlessly on one hot link. That dynamic path selection is exactly what municipal operators need when epidemiology or outbreak-model jobs suddenly spike fabric load.

InfiniBand congestion control with FECN marking fails because it only marks congestion and throttles injection rates; it does not reselect alternate LID paths or move training flows off a saturated spine. InfiniBand fat-tree routing tables refreshed by subnet manager rescans fail because rescanned fat-tree tables still install fixed deterministic routes and do not dynamically detour around a congested spine when credit starvation appears. InfiniBand Service Levels with Virtual Lane arbitration fails because VL arbitration and service-level priority only schedule traffic on the existing path; they do not cause dynamic path reselection when congestion appears.

Exam caveat: adaptive routing helps when alternate paths exist and the fabric is properly managed; it is not a substitute for capacity planning or correct QoS when interactive inference must outrank batch training. Operational check: in UFM or SM tooling, confirm adaptive routing is enabled for the public-health partition, inject or observe congestion on a known spine, and verify that training flows move onto alternate paths while PKeys and service levels remain intact.