Causal Masking
Masking technique preventing attention to future positions in autoregressive models.
In italiano: Causal MaskingEnsures token at position i can only attend to positions ≤ i. Essential for GPT-style models. Implemented as triangular mask.
Examples
- GPT autoregressive generation
- Decoder self-attention
- Preventing future leakage