A canal-lock keeper’s training curve either jumps wildly after each update or barely descends over many passes. What learning-rate idea best explains those two behaviors?
Select an answer to reveal the explanation.
Short Explanation
Step size is like how hard you turn the lock wheel: too hard and the water sloshes past the mark; too gentle and the level hardly changes. Large rates overshoot; tiny rates crawl. That is a conceptual learning-rate story, not a kernel rewrite.
Full Explanation
Learning rate controls how far each optimizer update moves the weights on a canal-lock keeper’s multimodal training curve. A rate that is too large can overshoot minima, producing wild jumps or divergence after updates. A rate that is too small yields updates so tiny that loss barely descends across many passes. Wild jumps do not by themselves prove the GPU is offline, nor does a crawl prove the dataset has no labels—those are separate infrastructure and data issues. Rewriting the loss in CUDA is not the first conceptual response to a rate problem. Learning rate is not a checkpoint renaming knob; it directly scales the update magnitude. Exam caveat: NCA-GENM expects the too-large / too-small learning-rate story without requiring scheduler mathematics. Operational check: for the civic lock-curve run, try a modest rate reduction when loss spikes chaotically, or a modest increase (or longer training) when loss plateaus near its start; always re-check with a held-out split. Learning-rate schedules can refine this picture later, but the first diagnosis remains step size too aggressive versus too timid on the lock curve. After each learning-rate change on the lock curve, compare a short window of steps rather than judging from a single noisy update. Learning-rate diagnosis on the lock curve is about update magnitude; fix labels and device health separately if those alarms fire. Keep a short LR change log next to the civic training chart.