Self-Attention

/sɛlf əˈtɛnʃən/

Self-Attention: An attention mechanism used in deep learning models that allows a neural network to weigh the importance of different parts of an input relative to each other. Self-attention is a fundamental technique in language and vision models. It allows models to consider relationships between elements within a single sequence, greatly improving comprehension and generation capabilities.

In italiano: Self AttentionLegge III · Come una rete impara dai dati

Self-attention is a fundamental technique in language and vision models. It allows models to consider relationships between elements within a single sequence, greatly improving comprehension and generation capabilities.

Examples

  • Machine translation models using self-attention
  • Recommendation systems
  • Text generation models

Fonti

Further reading

Self-Attention: why the idea of "stochastic parrots" no longer holds up