Causal Masking

Masking technique preventing attention to future positions in autoregressive models.

In italiano: Causal Masking

Ensures token at position i can only attend to positions ≤ i. Essential for GPT-style models. Implemented as triangular mask.

Examples

  • GPT autoregressive generation
  • Decoder self-attention
  • Preventing future leakage