Post-training quantization
Post-training quantization: Quantization applied after training to reduce inference memory and cost. It calibrates scales and zero-points on weights and, when required, activations without retraining. A representative dataset and task evaluation reveal precision loss. Examples include INT8 after training, Calibration on real examples, Quantized GGUF.
In italiano: Post-Training QuantizationLegge IV · Loss, gradiente e ottimizzatoriIt calibrates scales and zero-points on weights and, when required, activations without retraining. A representative dataset and task evaluation reveal precision loss.
Examples
- INT8 after training
- Calibration on real examples
- Quantized GGUF