Automatic differentiation
Automatic differentiation: Systematic calculation of exact program derivatives by applying rules to elementary operations in the computational graph. It is neither numerical nor purely symbolic differentiation. Reverse mode is efficient for scalar objectives with many parameters, such as neural losses.
In italiano: Differenziazione AutomaticaLegge IV · Loss, gradiente e ottimizzatoriIt is neither numerical nor purely symbolic differentiation. Reverse mode is efficient for scalar objectives with many parameters, such as neural losses.