Next Sentence Prediction
Next Sentence Prediction: Pre-training task predicting whether sentence B follows sentence A. Binary classification: 50% consecutive, 50% random pairs. Originally used in BERT but later found less useful than MLM alone. Examples include BERT pre-training task, Sentence relationship, Discourse understanding.
In italiano: Next Sentence PredictionLegge III · Come una rete impara dai datiBinary classification: 50% consecutive, 50% random pairs. Originally used in BERT but later found less useful than MLM alone.
Examples
- BERT pre-training task
- Sentence relationship
- Discourse understanding