Model collapse
Model collapse: The progressive degradation of a generative model trained on synthetic data produced by previous generations. As generated data progressively replaces real data, errors and underrepresented regions can be amplified. Severity depends on filtering, the share of synthetic data and preservation of the original distribution.
In italiano: Model CollapseLegge V · Overfitting, bias e collasso dei modelliAs generated data progressively replaces real data, errors and underrepresented regions can be amplified. Severity depends on filtering, the share of synthetic data and preservation of the original distribution.
Examples
- Loss of lexical diversity across model generations
- Vanishing tails of the data distribution
- Synthetic datasets accumulated without real data