Self-Attention
/sɛlf əˈtɛnʃən/
An attention mechanism used in deep learning models that allows a neural network to weigh the importance of different parts of an input relative to each other.
In italiano: Self AttentionSelf-attention is a fundamental technique in language and vision models. It allows models to consider relationships between elements within a single sequence, greatly improving comprehension and generation capabilities.
Examples
- Machine translation models using self-attention
- Recommendation systems
- Text generation models