Focal Loss
Focal Loss: Focal loss is a modified cross-entropy objective for classification with severe class imbalance. It reduces the contribution of examples the model already classifies confidently and correctly. For the probability pt assigned to the true class, the factor (1 − pt)^γ shrinks the loss on easy examples.
In italiano: Focal LossLegge III · Come una rete impara dai datiFor the probability p_t assigned to the true class, the factor (1 − p_t)^γ shrinks the loss on easy examples. γ controls how strongly this down-weighting increases with confidence, while α_t rebalances classes. With γ = 0 and uniform α, the objective reduces to standard cross-entropy.
Formula
FL(p_t) = −α_t(1 − p_t)^γ log(p_t)
Numerical example
import math
ce = -math.log(0.9) # true-class probability p_t = 0.9
gamma = 2.0
focal = (1 - 0.9) ** gamma * ce
print(round(ce, 4)) # 0.1054
print(round(focal, 6)) # 0.001054When to use it
Use focal loss when many easy negatives dominate training, as in dense object detection, or when rare positive examples need greater influence on the gradient.
Limitations
Focal loss does not fix noisy labels or guarantee better calibrated probabilities. γ and α require validation, and strongly suppressing easy examples can discard useful learning signal in moderately imbalanced datasets.
Examples
- RetinaNet object detection
- Imbalanced classification
- Hard example mining