County assessor parcel-description fine-tuning OOMs mid-epoch on fixed GPUs. Which change keeps the effective batch workable without buying hardware?
Select an answer to reveal the explanation.
Short Explanation
Think of accumulation like pouring coffee into a travel mug a little at a time—same total sip, smaller pours. Cut the microbatch so you fit, stack grads across steps, and the assessor job finishes without a shopping spree for GPUs.
Full Explanation
County assessor parcel-description fine-tuning that OOMs mid-epoch on a fixed GPU pool is usually a peak-activation and optimizer-state problem, not a shortage of checkpoint files. Cutting the per-step microbatch lowers the instantaneous activation and workspace footprint so the step fits in VRAM; raising gradient accumulation then sums gradients across those smaller steps so the optimizer still sees roughly the same effective batch size and similar update noise. That keeps learning dynamics workable without purchasing additional accelerators. Deleting checkpoints every step does not free the live training tensors that actually caused the OOM, and it destroys recovery points the assessor office needs when overnight jobs fail. Forcing full-precision FP32 for all layers increases, rather than decreases, weight, activation, and gradient memory, so it is the wrong direction when the allocator is already exhausted. Increasing the per-step microbatch until the GPU is completely full pushes peak usage higher and makes mid-epoch crashes more likely, especially once fragmentation appears. Exam caveat: accumulation preserves effective batch only when loss scaling and optimizer step cadence stay consistent with the new accumulation count—blindly changing one without the other can shift convergence. Operational check: log peak allocated bytes and successful steps-per-optimizer-update before and after the change; confirm OOM disappears while tokens-per-update remain in the intended band for the parcel corpus.