Next Sentence Prediction
Pre-training task predicting whether sentence B follows sentence A.
In italiano: Next Sentence PredictionBinary classification: 50% consecutive, 50% random pairs. Originally used in BERT but later found less useful than MLM alone.
Examples
- BERT pre-training task
- Sentence relationship
- Discourse understanding