Self-Attention
/sɛlf əˈtɛnʃən/
Self-Attention: An attention mechanism used in deep learning models that allows a neural network to weigh the importance of different parts of an input relative to each other. Self-attention is a fundamental technique in language and vision models. It allows models to consider relationships between elements within a single sequence, greatly improving comprehension and generation capabilities.
In italiano: Self AttentionLegge III · Come una rete impara dai datiSelf-attention is a fundamental technique in language and vision models. It allows models to consider relationships between elements within a single sequence, greatly improving comprehension and generation capabilities.
Examples
- Machine translation models using self-attention
- Recommendation systems
- Text generation models
Fonti
Related terms
Further reading
Self-Attention: why the idea of "stochastic parrots" no longer holds up