Momentum
/moʊˈmɛntəm/
Momentum: An optimization technique that accelerates gradient descent by accumulating a velocity vector in directions of persistent reduction in the loss. Momentum helps navigate ravines and accelerates convergence. It adds a fraction of the previous update vector to the current one, smoothing optimization trajectory and reducing oscillations.
In italiano: MomentumLegge IV · Loss, gradiente e ottimizzatoriMomentum helps navigate ravines and accelerates convergence. It adds a fraction of the previous update vector to the current one, smoothing optimization trajectory and reducing oscillations.
Examples
- SGD with momentum 0.9
- Nesterov momentum
- Accelerating convergence