Focal Loss

Focal Loss: Focal loss is a modified cross-entropy objective for classification with severe class imbalance. It reduces the contribution of examples the model already classifies confidently and correctly. For the probability pt assigned to the true class, the factor (1 − pt)^γ shrinks the loss on easy examples.

In italiano: Focal LossLegge III · Come una rete impara dai dati

For the probability p_t assigned to the true class, the factor (1 − p_t)^γ shrinks the loss on easy examples. γ controls how strongly this down-weighting increases with confidence, while α_t rebalances classes. With γ = 0 and uniform α, the objective reduces to standard cross-entropy.

Formula

FL(p_t) = −α_t(1 − p_t)^γ log(p_t)

Numerical example

import math

ce = -math.log(0.9)  # true-class probability p_t = 0.9
gamma = 2.0
focal = (1 - 0.9) ** gamma * ce
print(round(ce, 4))     # 0.1054
print(round(focal, 6))  # 0.001054

When to use it

Use focal loss when many easy negatives dominate training, as in dense object detection, or when rare positive examples need greater influence on the gradient.

Limitations

Focal loss does not fix noisy labels or guarantee better calibrated probabilities. γ and α require validation, and strongly suppressing easy examples can discard useful learning signal in moderately imbalanced datasets.

Examples

  • RetinaNet object detection
  • Imbalanced classification
  • Hard example mining

Fonti