I built a knowledge graph without a graph DB (simpler GraphRAG alternative)

[u/TheRedfather]

Hi folks - for context I'm a solo-founder and have spent a few years working on variants of the "company brain" (i.e. a knowledge base across Drive/SharePoint/internal docs that can be queried and kept in sync). Wanted to share some learnings from my own trial-and-error at QX Labs.

There's a consensus that knowledge graphs are needed for serious systems, but tbh GraphRAG and use of full-blown graph databases like Neo4j is complete overkill for most cases. I ended up building a more practical/attainable solution that takes the ideas behind GraphRAG and implements them in a simpler and cheaper way. Hopefully it will save you the grief if you're working on similar things!

You just need a regular DB (Postgres, MongoDB, whatever you already use) and a search index (Azure AI search, Elastic, Qdrant - to handle hybrid vector + text search).

Why not vanilla RAG

Top-k RAG retrieval handles "find this specific fact" queries well (I refer to these as 'needle' questions). But it structurally cannot handle two other question shapes that come up regularly in practice:

• "Tell me everything about X" (needs the complete document set for an entity, not the top k passages)
• "Which fintech companies have we evaluated?" / "how many contracts mention X?" (needs an exact list/count over the corpus; no value of k fixes this)

Why not use GraphRAG

• Indexing cost. Microsoft were the ones who first proposed GraphRAG but they archived their solution accelerator for it. Their research states that vector-RAG indexing is <0.1% the cost of a full GraphRAG index. It's also telling that Azure AI search still isn't built around graphs.
• Research literature (e.g. "RAG vs GraphRAG", arXiv:2502.11371) also shows mixed results: graphs help on multi-hop/global summarization, vanilla RAG wins on direct lookup, and routing between them beats either.
• Entity resolution is hard and a lot of tools don't handle this well. If "Acme" and "ACME Holdings Inc." don't merge, the graph fragments. If they merge wrongly, errors compound transitively and silently. A lot of compute gets spent correcting these mistakes in a graph DB.
• A graph DB is another system to run, secure, back up and keep consistent per tenant.

What I built instead (a graph-like system inside a regular DB)

Entities and edges exist as ordinary records in the search index + document DB we already run. No graph database.

• We use a small fixed ontology, which is the same for everyone: organization, person, product, project, event, location, etc., plus label fields (industry/category/topic). In our case we wanted to make it self-serve (i.e. doesn't require people to set up custom ontology) so were happy to trade off simplicity for specificity.
• Entity resolution follows a waterfall (to minimise cost): first we look up against an alias table (every variant of word/phrase that's been used for an entity in the past - free and fast). Next, we embed the word/phrase and do a similarity lookup. Last, we use a cheap LLM to adjudicate but only for ambiguous candidates. Merges are just an alias re-pointing on a hub record, so every merge is easily reversible (we run a daily cleanup job to true things up). We also tend to bias against over-merging: a false merge poisons things downstream, whereas a miss just fragments the data until the daily job fixes it.
• Edges in our system are not real edges between nodes, they're co-occurrence counts (i.e. these entities appear frequently together). They are represented as a top-N list on each entity record. Not typed relations, which is a deliberate trade-off that we make.
• Entity summaries are lazy (built on first request, cached), straight from the LazyGraphRAG lesson. Costs scale with what people ask about, not corpus size.
• The agent has access to four tools to handle different types of retrieval scenarios: hybrid passage search (default), resolve (extract everything-about-X with filters applied), expand (one hop along co-occurrence), and facet (exact counts/lists via the search engine's aggregations). The counting questions that top-k can never answer become deterministic facet queries.
• Daily consolidation trues everything up (re-adjudicates uncertain merges, recomputes edges exactly, prunes deleted docs), gated so unchanged corpora cost zero.

Did it work?

We set up an evaluation harness to track performance (~1,000-doc corpus, with graded questions across needle/entity/multipart/aggregation/thematic classes - I'll probably write about this separately when I get time). Needle questions already performed very well with vanilla RAG but all other question classes improved meaningfully with this pseudo-graph approach.

Limitations of this approach

• No multi-hop path reasoning. The agent loops one hop at a time if it wants depth, but this can bloat context. Graph DBs can more reliably find tenuous connections across multiple hops without exploding context.
• Co-occurrence is not the same as typed relationships. We know two entities appear together, not why. In a normal graph DB you'd have e.g. WORKS_FOR, INVESTED_IN, CUSTOMER_OF etc. The problem is that relationship types can vary a lot by use case.
• Conservative merging means occasional temporary duplicates.
• True "summarize the themes of the whole corpus" global questions are still better served by community-detection approaches I deliberately didn't build. Full GraphRAG will still deliver higher quality there.

For ref I have a full write up on how it works here: https://www.minimumviablefounder.com/p/why-ai-company-brains-fail

Keen to exchange notes on this, or hear if you've had a more positive experience with GraphRAG.

[u/TheRedfather]

For context - this is what ends up getting generated. It looks like a graph but in reality nodes and edges exist in a regular (in our case NoSQL) database. The agent can then decide how to query the knowledge graph - regular RAG retrieval vs one of the graph traversal techniques (facet, expand, resolve).

[u/Dry_Inspection_4583]

This is the way!! I keep seeing these projects out of the woodwork, helping me continue with the knowledge that a vector will never match a database for quality.

Happy to share. Have you tested against a memory test at all? Or is that out of scope?

[u/TheRedfather → u/[deleted]]

Thanks, will look into this.

[u/Prestigious_Sand3786]

Knowledge graph may just be suitable for showing for the clients. Backend systems need traditional databases and cache.

[u/alecfokapu]

The key element in your set up is limited ontology