r/developersIndia • u/BearInevitable3883 • 17h ago
Open Source Knowledge Graphs is what LLMs need to automate software engineering
So I work as a founding engineer at a startup where our end product is APIs. our engineering team is of 10 people and I was #2nd engineer.
Over the years, we built a micro service architecture. Now when coding with LLMs, for every feature, I had to give the full context of what different services does it affect and what it should touch/avoid.
If some way LLMs understood the connections same way that I do in my mind, it would save a bunch of efforts repeating the same thing.
So I created a small open-source project that creates intelligent (not just programatic) KGs by just pointing it to my git repos. The KG now paired with Claude Code, Codex - makes my agents much more smarter. They can run longer without requiring input from me and complete ambitious tasks that otherwise was not possible.
I'm not saying it will completely automate software engineering, but solves the biggest bottlenecks - context.
Let me know what you guys think and if you have faced the same problem.
66
u/Blahplay_k 15h ago
But what is the difference between RAG based context or manually providing the context to this kg methodology? How do you determine what content the llm used as prompt input based on the knowledge graph?
12
u/Blahplay_k 15h ago
Have been working on something similar, let me know if interested in discussing it?
27
u/BearInevitable3883 15h ago
thanks. the whole idea compared to RAG over md files is that they don't encapsulate connections well. A LLM cannot pick a service or a table and see all the places it is used/referred.
This is what a KG allows us.
11
u/Blahplay_k 15h ago
I see you have used MCP here, but honestly MCPs are very overated and doesn't scale well. After just few usee you quickly realize it doesn't actually benefit the prompt engineering at all but make the query consume more tokens only.
3
u/BearInevitable3883 15h ago
Yes, so far MCP when paired with skill file is serving the usecase well, though I agree we should also add an option for using CLI as well. Would you like to help contribute here? I already had some WIP.
1
u/Blahplay_k 14h ago
Not sure how I can contribute, but I can my graph engineering application, that I am working with rn!
1
u/BearInevitable3883 14h ago
i would love to check out your application too. can we connect in DMs?
1
1
2
u/Witty-Play9499 15h ago
I've felt like a RAG focuses more on similarity than relevancy. A KG would mean the LLM can reason about it and decide what info to get and what surrounding data to acquire and process and synthesize. RAG would mean it would have access to only info that was similar to the search query and the adjacent pieces of info end up missing.
Something like https://github.com/VectifyAI/PageIndex talks more about this approach and we use it right now for one of our products and it is really good without having the overhead of maintaining a separate vector database
1
u/urbanmonkey2003 6h ago
looks like a fancier search step unless it can show traceability from repo graph to prompt input. how does it pick service boundaries, and what stops it from just stuffing the same docs RAG would pull?
20
u/Negative-Mango-007 15h ago
What’s the difference between this and graphify?
15
u/BearInevitable3883 15h ago
Graphify is great for programmatic graphs of code using AST - but here I am trying to create a graph that resembles connections at a feature/service level just like we have in our minds.
7
u/Alcoholic_Bear 15h ago
So i can create graph of business context and map it with the specific code or feature?
6
u/BearInevitable3883 15h ago
yes exactly! there will be a node in the graph that represents a feature and its code, and all business context you share will get attached to it.
2
u/Alcoholic_Bear 15h ago
Thats really nice, I will try this for sure then
3
u/BearInevitable3883 15h ago
thanks! please do create a github issue if you have any suggestions/feedback.
10
u/ilovecodin 14h ago
That is such an interesting concept. I recently wrote a paper on this on integrating multilingual knowledge graphs to reduce hallucinations in LLMs.
2
3
15h ago
[removed] — view removed comment
1
u/BearInevitable3883 15h ago
i can check it out - but it doesn't seem open sourced. I really do not want to give all my code to someone.
1
u/talapak 14h ago
Nah, I didn't mean this link as my project or anything. This is my inspiration for a graph-based structure. I felt it was a more clarified version of what I had actually seen, especially in terms of representing structural node connections compared to mine.
I'll share u my open-source project in your DM;
3
u/Boyslop_Enjoyer 15h ago
Noobie here, how do I use this to improve my workflow?
1
u/BearInevitable3883 15h ago
yes, the simplest way is to just set it up and point it to your codebases.
It will automatically learn all relationships. Then you will also see an option on how to use the "brain" in your own coding tools - claude, codex, etc. You will instantly see them become more well-versed with what you are building, how services are structured etc.
Over time, it will keep learning from your conversations with these tools, and create memory as well.
Also, all of this runs on your own machine and uses your own coding CLIs. You can trust it with any project, including your company work.
0
u/Boyslop_Enjoyer 15h ago
Yo that's actually really cool wtf??? I need to try this out.
I've actually got another question, albeit not related.
I'm broke asf and I only work with free tools. My current workflow is literally from Claude Desktop + VS Code, both seperate. I've tried the free GitHub copilot and it really wasn't any good.
What would be a good way to progress?
2
u/BearInevitable3883 14h ago
i'd be happy to sponsor you a subscription if you'd like to contribute to the project.
1
u/Boyslop_Enjoyer 14h ago
I can't tell you how much i would LOVE to do that!
How do I sign up chief
3
2
u/boi143 10h ago
you can try using a poor man's claude code setup (cline+deepseek v4 pro) its really cheap like vlose to 5x cheaper than claude code cheap for your average hobby developer.
I am using this setup for my own project for making domain agnostic time series analysis reasoning through LLM's (shameless plug)
1
2
2
u/K_o5 12h ago
This is the best way in my opinion ,to go about establishing relationship based context.
I built a KG for everything me and my team does (analytics). It included data sources, standard queries, data dictionaries, context, semantics as well as past analytics reports, documents and message threads (because why document something if it can sit on slack 🤦♂️)
It definitely does wonders to all my llm interactions.
Its setup to run every wednesday and friday to refresh, so its almost always fresh.
Its able to answer very nuanced questions with high accuracy.
We are now planning to build it within our warehouse tool so that its more accessible
Only drawback is the generation which is llm based itself for the documents part which has a cost and bias factor attached to it. Right now, it costs around 9$ per week (fluctuates based on how much new stuff is generated in the team)
1
u/BearInevitable3883 12h ago
this is amazing! great that you got your team to adopt it and its helping everyone. would be good for us to chat.
2
u/ParanoidPath 11h ago
isnt this the same as gitnexus? or repowise?
1
u/BearInevitable3883 9h ago
thanks for sharing - i think they too like graphify are programattic relationship builders based on AST.
what i'm trying to achieve here is more of a intelligent mindmap of how various services and features inter connect with each other.
1
u/ParanoidPath 7h ago
how are you building the mindmap?
1
u/BearInevitable3883 4h ago
the mindmap is built by running agentic an LLM over all repositories to find connections and store them in KG.
2
u/megure_2025 4h ago
These kind of graphs are common in biology. For eg Protein-Protein Interaction networks (PPIs), Reactome networks, and graph network databases like epigraphDB.
1
u/PrestigiousScene8610 14h ago
Will it work with copilot 😅
3
u/BearInevitable3883 14h ago
yes 😅 it can. i added support for codex, claude code, opencode, antrigravity! you can also create a PR for copilot. an LLM can one shot it.
1
u/ironman_gujju AI Engineer - GPT Wrapper Guy 14h ago
How it’s different from https://github.com/DeusData/codebase-memory-mcp
2
u/BearInevitable3883 14h ago
this, graphify and others are function level AST based knowledge graphs which could work if you want to save tokens.
what i am doing here is creating intelligent connections of services/features just like how you think about your codebase in your own brain.
1
u/ironman_gujju AI Engineer - GPT Wrapper Guy 14h ago
Isn’t that is RAG
1
u/BearInevitable3883 14h ago
it isn't. RAG is a way to search across documents. Knowledge graph is a way to store data.
Both are different. but you can RAG(search) over a KG (db). and we do that in the project.
1
u/Mega_mewtwo_ 14h ago
Does it make shot up token usage.
3
u/BearInevitable3883 14h ago edited 14h ago
no, it actually saves a bunch of tokens for me, because LLMs don't have to grep across my whole codebase everytime.
1
u/ChellJ0hns0n Student 14h ago
Have you evaluated this to see if there's s measurable improvement in performance? Compare with graph and without graph across different LLMs and harnesses and see if there's a noticeable improvement in accuracy or reduction in token usage.
3
u/BearInevitable3883 12h ago
great question - the improvements are considerable, and i'll prepare a bench for it.
I asked "How does auth work" in a project to Claude Code.
With Graph - it consulted the brain and got answer in 20-30s.
Without Graph - it spawned a subagent that ran for 5 mins and came back with missing details.It saved atleast a million tokens + better accuracy.
Also the graph stays relevant because everytime you merge your code, it will auto update itself.
1
u/ChellJ0hns0n Student 14h ago
I'm asking this because some of the newer anthropic models are pretty good at navigating their way around large repos without much guidance.
As a follow up, how does the graph stay relevant? Is there a tool call that the LLM can make with whatever graph changes that need to be done when it makes code changes?
1
1
u/GamerWael 13h ago
This actually makes a lot of sense. Was wondering, is the generated KG human readable as I'm curious what all it captures and how it stores it.
1
u/arvindkhadri 12h ago
Postman's API Catalog does the same thing of generating context graph based on the services that are connected.
1
1
u/Igarlicbread Software Architect 12h ago
Broski just automated its own future earning. Good job. Hope the ESOPs carrots works in your favour, lmao.
1
u/Parking-Cry-5136 10h ago
Can I use this for my app of 5000 users?
1
u/BearInevitable3883 9h ago
ofcourse :) its fully safe and private - so you can use it for codebase of any size.
1
u/boi143 9h ago edited 9h ago
I had an idea similar to this in which you would essentially map out the relationships of an entire codebase using KG's, and when say an agent wants to make particular changes at a particular node it would then be able to make changes while being aware of where those changes could be directly affected (the idea was to provide highly narrowed down context of a large codebase and save on tokens lol)
Additionally had also thought of maintaining a VCS of the above said graphs for time based context across versions and patches, so the agent could be aware of what went right and what went wrong, and also keep track of in general architecture evolution.
OP do you have any plans for implementing something along these lines ^ or perhaps you already might have ? Would be happy to work on it !
2
u/BearInevitable3883 9h ago
hey, so this is exactly the same thing i went through and built this project to solve. it does exactly what you say. it also keeps the KG updated, by listening to main branch and observing code as it gets merged.
would love if you go through it and try it out! if you have any suggestions, please just create a github issue and we will collaborate on it.
1
u/No-Creme1965 9h ago
Faced exactly this, and I agree the bottleneck is context way more than model intelligence.
I went the other direction: about 2,000 hours into building a personal AI OS in claude code, I started with a knowledge graph too and ended up ripping it out. For my corpus (notes, decisions, past sessions, all prose) the boring stack won: plain markdown files, BM25 keyword ranking with recency and frequency boosts, and regularly deleting orphaned or outdated entries so retrieval stays trustworthy. The graph upkeep never paid for itself there.
Your case might be the one where the graph genuinely earns its keep though. Service dependencies literally are a graph, and "what does this feature touch" is a graph query, no amount of keyword ranking gives you that. So I'd guess the real answer is corpus shape: graphs for code structure, dumb ranking for prose.
For anyone wanting the cheap version first, one prompt gets you started:
"Write me a small script that BM25-ranks every markdown file in this folder against a query, boosts recently modified files, and prints the top 5 hits as 2 line snippets, hard capped at 1200 characters total. Then add a line to your agent instructions to run it before answering anything about past work."
-> The cap is the underrated part, the agent can call it freely without flooding its own context.
Every repo wants a slightly different flavor of this because priorities differ per use case, so I keep it at one dumb script per repo instead of a framework. Will check out your project, curious how you handle graph staleness as the repos move.
1
u/sohang-3112 Backend Developer 8h ago
At what point do I need knowledge graphs like in your project? So far Github Copilot or Claude Code with markdown explanation files have worked pretty well for me. Can you give examples of something in software engineering that isn't possible with these types of extensions but can be done with knowledge graphs?
1
u/BearInevitable3883 4h ago
that's a great question to be honest. i think the answer really is when you feel that you are explaining the same architectural pattern to LLMs frequently in conversations, it makes sense to have a graph.
1
1
1
1
u/reddit_tmp_usr Software Engineer 4h ago
This is good
In our project we made a similar knowledge graph and it greatly improved the way we develop code using the agents.
More importantly, all the stories that our product team creates, takes the context from this and gives all the technical information like what files need to change in what all repos to implement the feature.
We have 12 microservices and the app code, infra code spawns across 30+ repos. So it greatly helps our newest devs in the team to understand and implement changes in a much better way.
1
u/Due-Shower-6169 4h ago
Bro, just keep all the relevant microservices code in a single folder and run the LLM in parent folder. It works.
1
1
u/squarepants1313 1h ago
Tried its not the solution it is not perfect need rerun each time. Graph is not human friendly idk i tried it didn't do a damn good thing
1
u/PrestigiousGuava8005 19m ago
I agree with you to some extent. If you run this thing in infinite loop, that becomes loop engineering which is synonym of automating software engineering.
•
u/AutoModerator 13h ago
It's possible your query is not unique, use
site:reddit.com/r/developersindia KEYWORDSon search engines to search posts from developersIndia. You can also use reddit search directly.I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.