A logistics team trains a network that scores late-delivery risk. Plain stochastic gradient descent uses one learning rate for every parameter. The team switches to a method that adapts each step from the gradient's first and second moments. Which method is that?
Select an answer to reveal the explanation.
Short Explanation
Plain stochastic gradient descent is one gas pedal for every wheel. You switch to Adam, which adapts each parameter's step from the gradient's first and second moments instead of that single rate. The trap is a smaller constant rate, which is still one rate applied to every parameter.
Full Explanation
Plain stochastic gradient descent subtracts one learning rate times the current gradient, and that rate is shared across parameters. Adam keeps a running estimate of the gradient's first moment and of its second moment and uses those estimates to scale the step for each parameter, so parameters with different gradient scales do not share one fixed rate. That is the switch the team described. A smaller constant rate is still plain stochastic gradient descent: one rate, applied to every parameter, with no moment estimates. A step that uses only the current mini-batch gradient has no memory of past gradients and therefore no first or second moment to adapt from. A warmup schedule can change a global rate across time and still broadcasts that single rate to every parameter, which is not a per-parameter scale from both moments. Exam caveat: adaptive steps do not remove the need to watch whether training is diverging; they change how the step is sized. Operational check: confirm the optimizer state stores first-moment and second-moment estimates, not a single global learning rate alone.