LoRA (Low-Rank Adaptation)
LoRA (Low-Rank Adaptation): Parameter-efficient fine-tuning adding trainable low-rank matrices to frozen weights. Adds small trainable matrices A and B where ΔW = BA. Reduces trainable parameters to 0.1-1% while maintaining performance. Enables efficient adapter-based fine-tuning. Examples include Fine-tuning LLMs with LoRA, Adapter modules, Multi-task learning.
In italiano: LoRA (Low-Rank Adaptation)Legge III · Come una rete impara dai datiAdds small trainable matrices A and B where ΔW = BA. Reduces trainable parameters to 0.1-1% while maintaining performance. Enables efficient adapter-based fine-tuning.
Examples
- Fine-tuning LLMs with LoRA
- Adapter modules
- Multi-task learning