A water-meter office sends three inspection notes of different lengths in one GPU call so the card is not woken three times. How does that batch become one rectangular tensor?
Select an answer to reveal the explanation.
Short Explanation
Three inspection notes of different lengths have to sit in one rectangle so the GPU wakes once. Pad the short ones and pass an attention mask so the pads get ignored. Gluing them into one super-string mixes the stories, and neither a warp pack nor a job queue is the batch shape.
Full Explanation
A batched forward pass needs one rectangular tensor, so shorter notes are padded to a shared length and an attention mask marks the real tokens. Concatenating three notes into one string mixes their meanings and still leaves a single sequence. A warp-level CUDA pack is a kernel lab, not associate batching. Uneven civic notes are an application-shape problem, not a facilities ticket.