Pruning

Pruning: Removing unnecessary weights/neurons to reduce model size and computational cost. Identifies and removes weights with small magnitudes. Can be structured (neurons/filters) or unstructured (individual weights). Often combined with fine-tuning. Examples include Weight pruning by magnitude, Structured filter pruning, Lottery ticket hypothesis.

In italiano: PruningLegge IV · Loss, gradiente e ottimizzatori

Identifies and removes weights with small magnitudes. Can be structured (neurons/filters) or unstructured (individual weights). Often combined with fine-tuning.

Examples

  • Weight pruning by magnitude
  • Structured filter pruning
  • Lottery ticket hypothesis