LoRA (Low-Rank Adaptation)
Parameter-efficient fine-tuning adding trainable low-rank matrices to frozen weights.
In italiano: LoRA (Low-Rank Adaptation)Adds small trainable matrices A and B where ΔW = BA. Reduces trainable parameters to 0.1-1% while maintaining performance. Enables efficient adapter-based fine-tuning.
Examples
- Fine-tuning LLMs with LoRA
- Adapter modules
- Multi-task learning