A youth-makerspace copies a TensorRT-LLM engine built for the county H100 onto a classroom laptop GPU and it will not load. Why, and what should they keep?
Select an answer to reveal the explanation.
Short Explanation
An engine built for the county H100 will not load on a classroom laptop GPU. Optimized engines are GPU-family specific. Keep the portable checkpoint and rebuild, or pick a NIM profile, for the new box. Engines are not portable like a checkpoint.
Full Explanation
A TensorRT-LLM engine is compiled for a GPU family, so an H100 engine will not load on a typical laptop GPU. The portable artifact is the checkpoint, plus a rebuild or a NIM profile for the new box. Treating the engine as if it were a portable checkpoint is the associate mistake. A training-rank map does not explain the load failure.