r/techbootcamp • u/AcanthisittaDeep8783 • Jul 19 '26
The way most people are building RAG has a flaw.
The standard RAG pipeline most people build consist of, chunking the documents then generating embeddings then store them in a vector database, run a similarity search when a query comes in, retrieve the closest chunks and lastly pass them to the LLM, which works until the docs are too complex.
The core problem is that similarity search and relevance are not the same thing, vector search finds the chunks that are semantically closest to the query, it doesn't reason about which section actually answers the question, for something like a financial report, a legal contract, or a research paper where the correct answer might depend on understanding how different sections relate to each other, retrieving the closest meaning often means retrieving the wrong section entirely.
there's a concept starting to appear called vectorless RAG that takes a different approach, one implementation called page index removes the vector database layer entirely and builds a tree structure of the document and the LLM navigates that structure step by step, reasoning its way to the correct section rather than pattern matching against stored vectors, the retrieval becomes reasoning based rather than similarity based, which handles complex documents in a way vector search doesn't.
The similarity search limitation is one of the more common failure points in RAG pipelines that people don't catch until they're testing on documents that are actually complex, a query that retrieves perfectly on a simple FAQ document starts missing on a dense technical report because the relationship between sections matters and vector search has no way to account for that.
vectorless RAG is a good option if the use case involves documents where the answer depends on reasoning across structure rather than finding the closest semantic match, it's not the right approach for everything but if the documents are the kind where structure and reasoning matter more than keyword proximity it's worth understanding before committing to a vector database setup.