Quantization
Quantization: Reducing precision of weights/activations to lower memory and computation. Converts 32-bit floats to 8-bit integers or lower. Reduces model size 4x with minimal accuracy loss. Essential for edge deployment. Examples include INT8 quantization, Mobile deployment, 4-bit LLM quantization.
In italiano: QuantizzazioneLegge IV · Loss, gradiente e ottimizzatoriConverts 32-bit floats to 8-bit integers or lower. Reduces model size 4x with minimal accuracy loss. Essential for edge deployment.
Examples
- INT8 quantization
- Mobile deployment
- 4-bit LLM quantization