311 intent fine-tuning saturates GPU memory as batch size rises. What adjustment correctly continues effective batch growth?
Select an answer to reveal the explanation.
Short Explanation
When the 311 fine-tune fills the GPU, don’t just jam a bigger batch in—accumulate gradients across steps. Same effective batch, friendlier memory.
Full Explanation
311 intent fine-tuning often saturates GPU memory as micro-batch size rises. The correct continuation is to raise batch size until memory saturates, then use gradient accumulation to simulate larger effective batches—preserving optimization statistics without fitting an impossibly large micro-batch in VRAM, which is standard practice under municipal GPU caps. Forcing larger batches by disabling gradient checkpointing and hoping OOMs vanish usually increases activation memory and worsens OOM risk instead of solving it. Setting batch size to one forever and never revisiting memory limits forgoes throughput and can hurt stability when larger effective batches are appropriate for noisy 311 label distributions. Storing activations on the municipal open-data portal instead of GPU memory is not a training-memory strategy and would destroy step latency while creating privacy and governance problems for resident tickets. Exam caveat: accumulation steps should divide the target global batch cleanly so civic runs remain reproducible across nodes and checkpoint resumes stay consistent. Operational check: find the max non-OOM micro-batch on the 311 GPU, set accumulation to reach the desired effective batch, and verify identical effective-batch math and converging intent F1 on the validation set before multi-day fine-tunes.