Water-utility overnight jobs need larger effective batches than activations allow. What technique helps?
Select an answer to reveal the explanation.
Short Explanation
Checkpoint activations, recompute on the backward, and the water job suddenly has room for a bigger effective batch overnight. Hoarding every activation does the opposite.
Full Explanation
Water-utility overnight jobs that need larger effective batches than activations allow should use activation checkpointing: selected layer activations are discarded in the forward pass and recomputed in the backward, freeing peak memory so a larger effective batch or accumulation window can fit on the GPU. That compute-for-memory trade is the standard answer when activation storage, not model weight storage alone, blocks the target batch. Storing every activation permanently maximizes memory use and prevents larger batches by design. Reducing global batch to one token for the entire night fits easily but destroys gradient signal quality and overnight training efficiency for utility forecasting or document models. Disabling checkpointing so recomputation never occurs preserves the high-memory path that already blocks larger batches. Exam caveat: checkpointing trades extra compute for lower peak memory; use it when memory, not raw FLOPS, limits the civic job’s effective batch. Operational check: enable activation checkpointing on transformer blocks for the water-utility model, raise effective batch or accumulation toward the overnight target, confirm loss behavior remains sane versus the small-batch baseline, and balance recompute overhead against the throughput gain from the larger batch before locking the schedule.