A transit lost-and-found page starts posting inferences the instant the Triton container process appears, and the first wave of lookups fail while weights still load. What should the client wait for?
Select an answer to reveal the explanation.
Short Explanation
The container process can be up while weights are still loading. Wait for Triton’s ready or health check, then send user traffic. A compile log, an NCCL vote, or a boot-splash caption is not the ready signal.
Full Explanation
Serving has a ready state: the process can exist before weights are loaded, so clients must wait on Triton’s health or ready check. Racing /infer the moment the container starts produces failed first requests. A CUDA compile log is not the public ready signal, and an NCCL rendezvous is a training-collective idea. Captioning a boot splash does not tell you the model can accept civic traffic.