Self-Supervised Learning
Self-Supervised Learning: Learning paradigm where models create supervision signal from unlabeled data. Predicts parts of input from other parts (masked LM, image inpainting, next frame). Enables pre-training on massive unlabeled datasets. Foundation of BERT, GPT. Examples include BERT masked language modeling, GPT next token prediction, Image rotation prediction.
In italiano: Self-Supervised LearningLegge III · Come una rete impara dai datiPredicts parts of input from other parts (masked LM, image inpainting, next frame). Enables pre-training on massive unlabeled datasets. Foundation of BERT, GPT.
Examples
- BERT masked language modeling
- GPT next token prediction
- Image rotation prediction