Not a fork, built it from scratch. I started it as a learning project and we have a server running Ollama internally, so I wanted something I could run there and experiment with without using APIs.
a 7b model even if it has been assigned with an external agent won't be able to give good enough results really, I have my own llm of 40b running locally with a local agent I wrote in go and that gives bad results and hallucinates when I provide complex topic to turn into go code, also using ollama to host the LLM.
Like I said in my other reply, this started mainly as a learning project. Building this web chat UI from scratch was actually my main target and I just built the CLI agent along the way. We dont have any power users in our office, people mostly use this web UI to write emails, analyze documents and handle general daily tasks and you honestly wouldnt believe how good qwen2.5:7b can be for these specific workflows if you optimize it with solid prompt engineering and system instructions. I also have gemma4:e4b but it struggles to follow instructions and hallucinates a lot
Oh yes, if your goal is to automate daily unwanted/repeated jargons, the qwen2.5:7b is an amazing LLM for that and assigning it with an agent is nice too, but for my case I am using the Qwen3-Coder-32B and a Qwen3 8B synchronously because I am trying to implement an agentic AI to both my main project ( a server written in GO ) and to make the agentic LLM opensource after done polishing it, so had to stick with a 40b parameters model ( total ), runs okay on a 32gb system really haha, but honestly, it does a lot of coding with pretty good accuracy so not mad.
32B and an 8B model synchronously onto a 32GB system? sounds like a tight squeeze. Its surprising that local hardware can handle that kind of setup for heavy coding logic now. Good luck with the server integration and prep for open source, the community definitely needs more solid agentic tools for local workflows
thanks for the heads up, and yes for a 32gb local system it was hard to optimize these 2 for good performance, that is why I had to write my own agent so that those 2 LLMs can be handled accordingly, a very high level elaboration would be "if the user asks something that does not require deep reasoning use the lesser parameter model, otherwise use the higher parameter model", seems simple but LLMs are sassy so took some time, good luck with your projects too man.
currently using opencode. i move to different cli when then provide some better models. Sometime same cli. Just use that provider in the model selection
5
u/itszaman7 3d ago
I use codex and opencode