The regional transit HQ H100 is memory-tight on a larger decoder. Official additional material names TensorRT-LLM converting weights toward FP8 on H100. What should the team choose?
Select an answer to reveal the explanation.
Short Explanation
The HQ H100 is memory-tight on a larger decoder. Official path: optimize toward FP8 with TensorRT-LLM on that H100, then serve. Do not write a kernel, invent a numeric format, drop to CPU INT2, or treat FP8 as a cuDF dtype.
Full Explanation
Official additional material names TensorRT-LLM converting weights toward FP8 on H100 as a serving-time optimize choice. On a memory-tight H100 the associate path is optimize to FP8, then serve. Writing a custom kernel or inventing a numeric format is out of scope. A RAPIDS tabular dtype and a CPU INT2 detour are the wrong stack.