When the cited paper came out it was the first looking at this. And even with just 10% of the original data the effects were mitigated.
Since, another paper found a mix of synthetic and organic data is the best performing mixture.
The quality of the models producing the synthetic data matters a lot.
Other research has found huge benefits in training models with synthetic data from SotA models.
Models are only getting better, meaning the quality of synthetic data will be improving.
It only leads to collapse if all organic data representing long tails of data variety disappear. Which hopefully throws water on x-risk doomers as AI killing humanity would currently be a murder suicide.
No, it won’t.
A number of things:
It only leads to collapse if all organic data representing long tails of data variety disappear. Which hopefully throws water on x-risk doomers as AI killing humanity would currently be a murder suicide.