Causal language modelling
Causal language modelling: An objective training a model to predict the next token using exclusively the preceding context. A causal mask prevents access to future tokens. It is typical GPT pretraining and supports autoregressive generation. This perspective connects the definition with practical behaviour in artificial intelligence systems.
In italiano: Causal Language ModelingLegge III · Come una rete impara dai datiA causal mask prevents access to future tokens. It is typical GPT pretraining and supports autoregressive generation.