A poll-worker help page piles up while a volunteer keeps a laptop predict.py loop open and pastes questions by hand. Why move the model onto Triton?
Select an answer to reveal the explanation.
Short Explanation
A paste-into-predict.py loop is not a live service. Triton is a serving process that takes concurrent HTTP or gRPC calls and can batch them. A notebook cell is not a gRPC mesh, and NCCL is not why two clerks can be served at once.
Full Explanation
Live civic traffic needs a serving process. Triton accepts concurrent HTTP or gRPC requests and can batch them; a paste-into-a-script loop cannot. A notebook cell is not a production mesh. NCCL is a training-collective stack, not the reason two clerks can be served. Multimodal design is a different certification and does not require leaving the model in an interactive script.