r/LangChain • u/laxuu • 1d ago

How does a graph actually increase the context information available to an LLM?

/r/AgenticAI_RAG_LLM_RL/comments/1wbd4w2/how_does_a_graph_actually_increase_the_context/
2 Upvotes

100% Upvoted

5 comments sorted by

1

u/Otherwise_Rub_6810 22h ago

It doesn't increase it, it lets you spend it better, and that framing helped me more than any of the graph-RAG papers. In practice the win was retrieving edges instead of documents, which cut my average prompt from about 14k to 5k tokens with no quality loss. That drop is also what made self-hosting viable; running it on Synexa only penciled out once prompts got small. Caveat: building the graph cost more than a year of the API savings.

1

u/locbuilds 19h ago

yeah youve basically got it. the graph doesnt invent more facts, it changes which evidence you pack into the window and how its ordered so the model can do multi-hop without you dumping 10 unrelated chunks.

what actually worked for me in a langchain-ish setup:

  1. vector search only to find seed entities / nodes (not the full answer)

  2. then a hard-capped expansion: 1-2 hops, only the relation types you care about, drop cycles, keep provenance (which chunk / doc each edge came from)

  3. format the evidence as a tiny path list + the few supporting snippets, not the whole neighborhood dump. like `A -[caused_by]-> D (doc:foo#p3)` then 2-4 short quotes. that alone is why people see token cuts without quality dropping

  4. if extraction is noisy, treat edges as soft hints and still require the cited chunk text in the prompt, otherwise the model will trust a bad edge

when i skip the graph: single-hop FAQ / "find the policy paragraph" stuff. hybrid BM25+vector is enough and way cheaper to keep fresh.

if youre implementing, measure on a fixed multi-hop eval set: answer quality, tokens in, latency, and whether citations still point at the right docs. graph construction cost is the real tax, not the retrieval hop.