Adagrad
Adagrad: Adaptive learning rate optimizer that adapts rates per parameter based on historical gradients. Accumulates squared gradients, larger accumulation means smaller learning rate. Good for sparse data. Can have diminishing learning rates over time. Examples include Sparse gradient optimization, NLP embeddings, Convex problems.
In italiano: AdagradLegge IV · Loss, gradiente e ottimizzatoriAccumulates squared gradients, larger accumulation means smaller learning rate. Good for sparse data. Can have diminishing learning rates over time.
Examples
- Sparse gradient optimization
- NLP embeddings
- Convex problems