A housing authority wonders why last night’s model-update job needed far more fabric bandwidth than today’s scoring service. What explains the difference?

The housing authority saw higher fabric bandwidth on last night’s model-update (training) job than on today’s scoring (inference) service because training commonly synchronizes large gradients and shuffles substantial training data among GPUs, while inference mainly serves smaller forward-pass requests for predictions. That bandwidth asymmetry is a standard municipal AI fabric observation when comparing overnight training to daytime scoring. Claiming training distributes large KV-cache segments across GPUs per step misplaces the bandwidth burden—KV-cache sharding is an inference-time decoding optimization for serving, while training's heavy fabric traffic comes from gradient synchronization and dataset shuffling rather than per-step KV-cache movement. Saying training writes large checkpoint snapshots to parallel storage each step confuses storage I/O with fabric collectives—checkpoints are periodic writes to parallel storage, while the update job's higher fabric use comes from per-step gradient synchronization and dataset movement, not storage snapshots. Saying training only syncs large augmented data batches across nodes each step is incomplete—it omits large gradient synchronization, which together with dataset movement is what makes training far heavier than smaller forward-pass inference requests. Exam caveat: poorly designed inference (huge batching, frequent model reloads, multi-GPU speculative decoding at scale) can raise serving fabric use, but the stem’s contrast—update job versus scoring service—still maps to training-heavy versus inference-lighter traffic. Operational check: compare fabric counters during training windows versus inference SLOs; size training networks for gradient exchange and data ingest, and size inference networks for concurrent forward-pass fan-out with separate capacity plans for each.