Cosine annealing
Cosine annealing: A schedule reducing learning rate along a cosine curve from a maximum towards a predefined minimum. It offers slow initial reduction, faster middle decay and a gentler ending. Restarts and multiple cycles require choices consistent with the training budget.
In italiano: Cosine AnnealingLegge IV · Loss, gradiente e ottimizzatoriIt offers slow initial reduction, faster middle decay and a gentler ending. Restarts and multiple cycles require choices consistent with the training budget.