A library kiosk must keep a summarizer interactive on a single workstation GPU. Which technique specifically targets activation memory?
Select an answer to reveal the explanation.
Short Explanation
Your kiosk GPU can run short on memory when activations grow. Activation quantization reduces the memory they use, helping your library summarizer stay interactive on one workstation GPU.
Full Explanation
Interactive library-kiosk summarizers on a single workstation GPU are often limited by activation memory—the attention and feed-forward network intermediates—rather than static weight size alone. Applying activation quantization lowers the bytes stored for those intermediates, helping peak memory stay within the kiosk GPU and preserving interactivity without necessarily requantizing every weight. Doubling the context length without changing precision increases the activation footprint and worsens the memory problem. Storing every activation in FP64 for numerical “safety” multiplies memory cost and is unnecessary for typical summarization inference on modern GPUs. Freezing the embedding table changes which parameters are updated, but frozen embeddings do not reduce peak activation memory during forward passes over long catalog blurbs. Exam caveat: activation quantization can interact with sequence length and batch size, so validate it on the longest kiosk queries and concurrent sessions expected during library opening hours. Practical check: profile peak allocated activation memory before and after quantization on the workstation GPU under a scripted multi-query load, then confirm p95 latency stays within the kiosk interactivity SLA.