Transit inference cannot wait in a general training queue. What Run:ai administration fits?
Select an answer to reveal the explanation.
Short Explanation
Pin transit inference to an always-on node pool—reserved lanes for live serving. Dumping it into preemptive training or CPU spot nodes makes ETA models late. Turning serving off for training peaks breaks riders.
Full Explanation
Transit inference serving riders cannot wait behind long training queues. Run:ai administration should pin transit inference to a node pool with always-on (non-preemptible) GPUs reserved for municipal serving so latency-critical pods stay scheduled even when training demand spikes. Dedicated or guaranteed pools separate serving SLOs from best-effort training. Mixing latency-critical inference into preemptive best-effort training only exposes ETA serving to eviction and queue delay. Shutting inference off during every training peak abandons the rider-facing service. Scheduling inference exclusively on CPU-only spot nodes usually misses GPU model latency/throughput needs and spot volatility. Exam caveat: always-on pools cost utilization headroom—size them to transit SLOs and keep training on separate preemptible capacity rather than over-provisioning everything as guaranteed. Operational check: deploy transit inference on the pinned pool, flood the cluster with training jobs, and confirm inference pods remain Running on the reserved GPUs without preemption. Label transit serving nodes clearly in Run:ai/Kubernetes and deny training jobs via taints/tolerations or pool placement constraints so overnight training cannot silently pack onto the always-on inference GPUs. Pinning is only effective when scheduling constraints enforce the pin under contention for municipal ETA serving.