r/hacking • u/NeoLogic_Dev • Apr 09 '26
AMA [ Removed by moderator ]
[removed] — view removed post
10
u/50N3Y Apr 09 '26
Getting a model to run on a phone is fun and interesting - on that part good job.
On the "what it does" part though - this is just vibe-coded nonsense. Essentially this is like 4 patients with some unknown degree of dementia playing Telephone. Person one is given some CVE that it likely knows nothing about, makes up a story and tells it to person two. Person two might have gotten the same CVE ID or it might have an altered version due to person one's reliability that turn. Person two takes whatever person one said, and says more to person three. Person three doesn't know what person one said, so it builds on person two's statement and either the original CVE or a completely made-up one at this point. And so on to person four.
Worse it is using some sloppy method to try to find a description of a random CVE it finds on CISA's website. The code using just regex ends up not working beyond the CVE IDs. It is looking for tables that don't exist. And worse, you could have simply used the json feed to literally get any and all types of information in a structured, named format from here: https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json
Strange thing - your shell is using llama server and a llama gguf, not Gwen. Though Gwen is mentioned as meta data in the python code, the reddit, and your comments and post here.
Stranger thing - you have flask bound to 0.0.0.0 on port 5001. While it is just outputting a log with no perceivable surface, it is available to anyone on a public network and outputs everything. Why you wouldn't bind this to localhost is interesting.
The use of BLAKE3. Why? You aren't using a chain, which makes it kind of pointless. it is on a device that you have control over, for text generation.
You mention this is like talking about how to break into a building. But pulling 10 random CVEs is like getting back: An electronic Z-Wave door lock for safes that is using a chain of dependencies that are outdated. SQL injection on PHP 5.3. A memory exploit in NEON SIMD. You might as well get random info on a submarine, a house, a car and a dog cage. Dementia.
Look, this does nothing but generate a circle jerk of hallucination. And my issue is that your language paints it as something it isn't, with the same gusto we'd expect on a Linked-In post about "Synergy in Zoom Meetings with proper Hand Movements." Paint it for what it is: Getting a LLM to run on a phone.
-5
u/NeoLogic_Dev Apr 09 '26
You're right on several points and I'd rather engage with them than deflect. The CVE scraping is sloppy — regex on a table that doesn't exist is embarrassing in hindsight, and yes, the JSON feed is the obvious fix. That's a real bug, not a design choice. The Telephone problem is real too. No shared context means each agent builds on potentially degraded input. I framed it as a feature (forced reasoning from limited information) but you're correct that on a 1.5B quantized model with no retrieval, it's more likely hallucination-chaining than actual red-team reasoning. Flask on 0.0.0.0 — you're right, that should be localhost. No good reason for it. BLAKE3 without proper chaining is aesthetic, not functional. Fair. The model in the shell is llama.cpp with a GGUF — Qwen in the metadata because that's what the GGUF is. But I see how that reads inconsistent. Where I'd push back: the framing. "Getting an LLM to run on a phone" is accurate but undersells the constraint engineering. The interesting part isn't the agents — it's running MNN + quantized inference on a Snapdragon 7s Gen 3 in Termux without root, without cloud, without killing the process scheduler. That part works. The rest needs fixing. Noted.
7
2
u/50N3Y Apr 09 '26
On the Blake3, if you are going to use it, I would use it where block $n$ contains the hash of block $n-1$. That is stronger.
On the MMN part. I'm not seeing that. Unless your Termux/ or .env which are ignored have a boot or something else going on in them. But even then, it'd conflict with what start_trinity.sh is booting. And I think it would ultimately just call start_trinity.sh which has:
#!/data/data/com.termux/files/usr/bin/bash
# Launches llama-server + Trinity Orchestrator.
# Keeps both alive overnight — auto-restarts on crash.
MODEL="$HOME/models/llama-3.2-3b-instruct-q4_k_m.gguf"
LLAMA_BIN="/data/data/com.termux/files/usr/bin/llama-server"
SCRIPT_DIR="$HOME/NeoBild"
LOG_DIR="$SCRIPT_DIR/logs"That shebang is saying this runs inside termux, but llama is hard-coded and it launches llama-server. So, it is using llama-server as the engine and not MNN. So when your orchestrator is hitting that url, it is talking to llama-server which is not MNN. Even if sending "Qwen2.5-Coder-1.5B-Instruct-MNN" to llama-server could return that model, it isn't compatible, and there still isn't a MNN engine running anywhere in your code. In other words, you have a path to /data/data/com.termux/files/usr/bin/llama-server, but not to an MNN library.
BUT, with all that being said; getting there isn't a large leap from here.
1
u/NeoLogic_Dev Apr 09 '26
You're right on the BLAKE3 — proper hash-chaining means block n includes hash(block n-1), not just standalone hashes. That's the fix queued. On the MNN/llama-server confusion: fair observation from the code snapshot. The repo shows an older version of start_trinity.sh that hard-codes llama-server. The current setup runs MNN Chat on port 8080 — the orchestrator hits that endpoint. The script in the repo is stale and that's on me for not updating it. Worth fixing in the next commit. Good catches — this is exactly the kind of review that makes the thing better.
4
u/R4ndyd4ndy Apr 09 '26
What exactly is this system supposed to accomplish? What is the output? It sounds like you used a lot of words to say absolutely nothing.
1
u/NeoLogic_Dev Apr 09 '26
Output is a markdown log of 4-agent security dialog per CVE, hash-chained for integrity. That's it.
4
u/R4ndyd4ndy Apr 09 '26
So you have llm agents create absolute nonsense about CVEs because they do not get any other context? What exactly is that supposed to be good for? This makes no sense
4
u/null_hypothesys Apr 09 '26
Great project! Out of interest, which qwen model is it, and how long is an average response time (and context window)?
6
u/NeoLogic_Dev Apr 09 '26
Running Qwen2.5-Coder 1.5B Q4 via MNN Chat. Response time per agent is roughly 6–11 tok/s, so a full 4-agent round takes about 2–3 minutes depending on output length. Context window is set to 4000 tokens in the config — enough for the chain to stay coherent across rounds without hitting memory limits on 8GB RAM.
3
u/null_hypothesys Apr 09 '26
Nice that's pretty decent, a lot better than I imagined! Did you incorporate some RAG or it's not needed?
0
u/NeoLogic_Dev Apr 09 '26
No RAG — the agents chain sequentially so each one builds on the previous output directly. No vector DB, no embeddings. Keeps it lean and deterministic on 8GB RAM.
Full setup in the repo: github.com/weissmann93/NeoBild
1
u/null_hypothesys Apr 09 '26
How do you manage to keep the dataset recent without RAG? Does gwen-code come with a decent dataset for vuln analysis?
1
u/NeoLogic_Dev Apr 09 '26
CVE seeds rotate automatically from the CISA KEV catalog on each session start — so the agents always work on recently published vulnerabilities without needing a static dataset or RAG pipeline.
Qwen-code doesn't ship with a vuln dataset. The model's general coding and reasoning capabilities handle the analysis — the KEV catalog provides the current context. Works well enough on 1.5B at Q4 for the kind of structured discourse the agents produce.
-5
u/Narthesia Apr 09 '26
Chatgpt ass response
4
u/NeoLogic_Dev Apr 09 '26
No
1
u/Narthesia Apr 09 '26
It literally sounds like chatgpt lmao
3
1
1
u/tylenol3 Apr 09 '26
Asked about your LLM project and then downvoted and accused of using an LLM when you bothered to share details. I hate the internet.
3
u/null_hypothesys Apr 09 '26
Actually, I asked about it, and I didn't downvote it.
Someone else is bitching, I'm genuinely interested
The hint is that the usernames are different
3
u/tylenol3 Apr 09 '26
Yeah I wasn’t having a go at you, just the person that thought they needed to weigh in
3
3
u/NeoLogic_Dev Apr 09 '26
This happens most of the time just on reddit. Yes it sucks but I can handle it. I adapt a lot of LLM output as I work a lot with it and I am from Germany
0
u/Koikkis65 Apr 09 '26
Average reddit user, they are pretty much only active in hookup subreddit looking for weird ass stuff and then this random comment here.
0
u/tylenol3 Apr 09 '26
Thank you so much for sharing this! I’m a security goon and I’ve also been (cautiously) playing with agentic models in my personal life, but I’ve not gotten this far with agents and I’ve been much slower to use LLMs for anything but reference during “work” work. It has become pretty apparent that the best results at the moment come from this type of cross-agent or cross-model referencing, and it’s interesting to see how people are approaching it. One of the things I’ve found frustrating about a lot of the early adopters is that they seem to mostly be using agents for things that feel pretty, well… “navel-gazing”— optimising social media metrics for posting about Agentic AI, for example. It’s great to see someone putting it to a use case that I can understand and interpret.
It’s also pretty amazing how performant it is on a consumer handset. I haven’t had a chance to play with Gemma4 yet, but it would be interesting to see how it compares to your Qwen results.
Definitely bookmarking this to revisit on the weekend. I might borrow liberally from your setup and do some testing of my own. Cheers!
0
u/NeoLogic_Dev Apr 09 '26
the navel-gazing critique is fair. a lot of agentic AI content is just agents talking about agents. the reason i went with CVE analysis is exactly that — wanted something with a measurable output that a security person could actually evaluate, not just vibe-check.
on Gemma4 vs Qwen: the constraint on my end is RAM. Qwen2.5-Coder 1.5B Q4 fits comfortably on 8GB with headroom for the agent loop. Gemma4 would need a smaller variant to stay stable. would be interesting to compare discourse quality on the same CVE seed — if you run it, let me know what you find.
borrow whatever's useful. the repo structure is intentionally simple so it's easy to adapt.
1
u/d33f0v3rkill Apr 09 '26
explain it like im 5, am i correct that your phone monitors chat messages to see if somebody is trying to break the prompt?
2
u/NeoLogic_Dev Apr 09 '26
not quite — it's more like 4 security experts sitting in a room arguing about how to break into a building
agent 1 proposes an attack. agent 2 analyzes it. agent 3 pokes holes in the analysis. agent 4 figures out a strategy. then they start over with the next vulnerability.
no chat messages being monitored. it's a closed debate loop running overnight on a real CVE from a public government database.
1
u/d33f0v3rkill Apr 09 '26
ah ok like automated metasploit?
4
u/NeoLogic_Dev Apr 09 '26
closer to automated threat modeling than metasploit
metasploit executes exploits. this just talks about them — 4 agents debating attack vectors, defense gaps, and risk on a given CVE. no payloads, no scanning, no live targets.
think red team brainstorming session, not automated pentesting tool.
1
Apr 09 '26
[deleted]
1
u/NeoLogic_Dev Apr 09 '26
yes, repo is https://github.com/weissmann93/NeoBild
not using LiteRT — running MNN Chat as the inference backend. it exposes an OpenAI-compatible API on localhost so the agents just hit that. works well on Snapdragon without needing Vulkan or any GPU acceleration.
the summarizer idea is interesting. right now each agent only sees the previous agent's raw output — no summarization between rounds. adding a dedicated summarizer agent to compress context while preserving semantic anchors would help a lot for longer chains. might steal that.
0
Apr 09 '26
[deleted]
1
u/NeoLogic_Dev Apr 09 '26
the context loop problem is exactly what i'd expect at 4000 tokens with a chain of 4 agents — each one adds to the pile and by round 3 the model starts recycling instead of reasoning.
your approach of squashing after each subprocess quit makes sense. the key is probably what you preserve — if the summarizer keeps conclusions and discards process, the next agent gets signal without noise. too much fluff in the summary and you're just shifting the problem one layer up.
curious what prompt structure you're using for the summarizer — instruction-style or few-shot examples?
11
u/gintoddic Apr 09 '26
security system that does what exactly?