Self-Supervised Learning
Learning paradigm where models create supervision signal from unlabeled data.
In italiano: Self-Supervised LearningPredicts parts of input from other parts (masked LM, image inpainting, next frame). Enables pre-training on massive unlabeled datasets. Foundation of BERT, GPT.
Examples
- BERT masked language modeling
- GPT next token prediction
- Image rotation prediction