ReLU (Rectified Linear Unit)
/ˈriːluː/
An activation function that outputs the input if positive, otherwise zero: f(x) = max(0, x).
In italiano: ReLU (Rectified Linear Unit)ReLU is the most popular activation function in deep learning due to its simplicity and effectiveness. It helps mitigate vanishing gradient problems and enables faster training compared to sigmoid or tanh.
Examples
- Hidden layers in CNNs
- Deep neural networks
- Most modern architectures