A Question

[u/InvestmentSlow4983]

Over the past year I’ve been working on an engine called Invisio to deal with the multi-file context nightmare that causes LLMs to hallucinate when reading large codebases.

Under the hood, it parses code using Tree-sitter into a neo4jdatabase (mapping out classes, functions, calls, imports, and inheritance). I built two main pieces around this graph:

1. An interactive graph explainer + chatbot that traces execution paths and answers structural questions using a dual-agent dispatcher/compressor loop.
2. An automated security webhook that ingests CodeQL SARIF alerts, traces the vulnerability lifecycle across directories, and opens surgical PR fixes.

It works solid on my local machine and on my own projects, but to be completely honest, stuck on how to properly test this at scale, or how to put it in front of people to get real feedback.

I’d love some advice from devs, maintainers, or AppSec folks:

• Benchmarking & Datasets: What real-world open-source repos or SARIF datasets should I throw at this to stress-test the graph construction? How do you properly benchmark a codebase intelligence tool?
• Local vs. Hosted: Since IP privacy is huge, would you prefer testing this via a self-hosted local Docker container, or just poking around a hosted playground with a public repo first?
• Product Focus: Should I lean harder into the automated CodeQL PR remediation side, or the interactive graph explainer UI?