Vector search has been treated as the default answer for LLM retrieval

RAG became almost synonymous with “the right way” to give a model access to external knowledge at inference time. For the last two years I've been working with LLM pipelines on enterprise data, living inside that assumption. So when I saw a paper from PwC, “Is Grep All You Need? How Agent Harnesses Reshape Agentic Search”, it stopped me. They ran 116 questions from the LongMemEval benchmark across four agent setups, testing grep and vector retrieval under two delivery modes: inline results injected into context, and file-based results the model reads separately (also an interesting way to deliver information to a model). The short result (and this matters: the grep advantage is most pronounced in CLI agent setups, custom pipelines with RAG still hold up well): grep outperformed vector on every harness-model pair when results were delivered inline. The largest gap was 86% vs 63% on the same corpus, same model. The same Claude Opus 4.6 scored 93% on one setup and 76% on another, without changing the retrieval method at all. What moved the numbers wasn't the retrieval strategy. It was the harness, the prompt construction, and how results were handed back to the model. Change the delivery mode from inline to file-based, and the ranking flips on half the configurations. The authors scope this carefully: LongMemEval rewards literal fact retrieval, exact dates, counts, specific phrases. Grep finds those without an embedding bottleneck. In domains where evidence is rarely verbatim, the picture may look different. But the broader point holds regardless of domain: retrieval strategy, agent scaffolding, and result delivery path are one system. Optimizing them independently will give you locally clean results and globally confusing behavior. That's the kind of reasoning that doesn't come from knowing tools. It comes from understanding how systems interact, and being willing to ask what else might be moving before you reach for the obvious fix. The paper: https://arxiv.org/abs/2605.15184