Activation Function
/ˌæktɪˈveɪʃən ˈfʌŋkʃən/
A mathematical function applied to a neuron's output to introduce non-linearity into the network.
Activation functions enable neural networks to learn complex patterns. Common functions include ReLU, sigmoid, tanh, and softmax. They determine whether a neuron should be activated based on input.
Examples
- ReLU for hidden layers
- Sigmoid for binary classification
- Softmax for multi-class classification