A cemetery-plot weekend import must embed two thousand short blurbs, while the visitor kiosk has one guest. How should batching be set?
Select an answer to reveal the explanation.
Short Explanation
Two thousand weekend blurbs can sit in a big batch. One kiosk guest cannot wait for that batch to fill. Heavy batch the import; keep the kiosk on small or dynamic batches. One-at-a-time overnight and a training all-reduce are not that policy.
Full Explanation
Batching policy follows traffic shape. Two thousand delay-tolerant blurbs should be grouped so the GPU stays busy. A single kiosk guest should be served with a small or dynamic batch so the request is not waiting for a large batch to fill. Processing the import strictly one-at-a-time wastes throughput, and a training collective is not a serving batch policy.