Gradient Descent
/ˈɡreɪdiənt dɪˈsɛnt/
An optimization algorithm that iteratively adjusts parameters to minimize a loss function by following the gradient.
Gradient descent updates parameters in the direction opposite to the gradient. Variants include batch, mini-batch, and stochastic gradient descent (SGD). Learning rate controls step size.
Examples
- Training neural network weights
- Optimizing linear regression
- Fine-tuning model parameters