Michele LaurelliBlog

Free tool · browser-only calculation

Activation function and derivative visualizer

Compare sigmoid, tanh, ReLU, GELU and SiLU with derivatives, saturation and interactive curves on a shared chart. With guided examples and shareable.

-4-224yx

    How the calculation works

    σ(x)=11+e−x,ReLU⁡(x)=max⁡(0,x),SiLU⁡(x)=xσ(x)\sigma(x)=\frac{1}{1+e^{-x}},\quad \operatorname{ReLU}(x)=\max(0,x),\quad \operatorname{SiLU}(x)=x\sigma(x)
    Formula as plain text\sigma(x)=\frac{1}{1+e^{-x}},\quad \operatorname{ReLU}(x)=\max(0,x),\quad \operatorname{SiLU}(x)=x\sigma(x)

    Activation functions introduce nonlinearity and affect gradient flow. Sigmoid and tanh bound their outputs but saturate for large inputs; ReLU is simple and does not saturate for positive values.

    GELU and SiLU are smooth functions widely used in modern models. A shared chart makes it easier to compare behaviour near zero, tails and derivatives without running a model.

    The classical ReLU derivative is undefined at zero and frameworks choose a subderivative convention. The chart is educational and does not replace tests on real architectures and data.