Learning Rate
/ˈlɜːrnɪŋ reɪt/
Learning Rate: A hyperparameter controlling how much model weights are updated during training. Learning rate determines the step size in gradient descent. Too high causes instability, too low slows training. Common strategies: fixed, decay, cyclical, adaptive (Adam). Examples include Learning rate 0.001 for Adam, Learning rate decay schedule, Warm-up then decay strategy.
Legge IV · Loss, gradiente e ottimizzatoriLearning rate determines the step size in gradient descent. Too high causes instability, too low slows training. Common strategies: fixed, decay, cyclical, adaptive (Adam).
Examples
- Learning rate 0.001 for Adam
- Learning rate decay schedule
- Warm-up then decay strategy