Next Sentence Prediction

Next Sentence Prediction: Pre-training task predicting whether sentence B follows sentence A. Binary classification: 50% consecutive, 50% random pairs. Originally used in BERT but later found less useful than MLM alone. Examples include BERT pre-training task, Sentence relationship, Discourse understanding.

In italiano: Next Sentence PredictionLegge III · Come una rete impara dai dati

Binary classification: 50% consecutive, 50% random pairs. Originally used in BERT but later found less useful than MLM alone.

Examples

  • BERT pre-training task
  • Sentence relationship
  • Discourse understanding