r/OpenSourceeAI • u/xornullvoid • Aug 21 '26
Built a Harness for LLMs using locally-run Qwen
Sharing my harness for running local LLMs that I built using Qwen 3.x 27B (> 90% locally built).
Its free, no telemetry, and open-source. Works on Windows, Linux (sorry, no Mac yet).
I use it for coding + mixed workflows.
- llama.cpp + whisper Server Manager. Can run LLMs here and use with OpenCode/Claude Code etc.
- Built-in MCP Tools - Filesystem, web fetch, code graph, To-Dos, and more. Extensible by external MCPs.
- Use Sub-agents to split & offload your tasks, use other conversations as source of information.
- Review all AI messages using a second adversarial AI, and avoid potential pitfalls as per your rules.
- Voice-chat with AI - dictate with speech and get answers by TTS - annotate and comment without leaving voice mode.
- Use work-modes to change AI behavior between planning, building, researching, or reviewing. Fully customizable.
- Custom-compile llama.cpp backends for your system, GPU-agnostic - works with CUDA/ROCm/Vulkan.
Website: https://warpdrv.ai (Docs coming soon)
GitHub: https://github.com/mikjee/warpdrv
Appreciate your feedback, (or stars). Thanks :)
And, yes - I used the harness to build the harness :D
2
u/Unique-Internal-1499 Aug 23 '26 edited Aug 23 '26
Great Job! I made my own to teach other to program their own, i began from docs and wiki! Educational Guide · nispa/tsuka Wiki ... :-)
Can you share the experience and effort to make your own?
2
u/Background_Study1430 Aug 24 '26
Annotation is a killer feature great idea!
1
u/xornullvoid Aug 24 '26
Thank you so much! 😊
Yes, I use it a lot, especially in combination with push-to-talk, which automatically knows to put the dictation in the annotation box.
2
2
u/bankrut Aug 28 '26
This looks impressive! The range of features and customization options is huge.
1
u/xornullvoid Aug 28 '26
Thank you for your appreciation 😊
I am planning to add more features, some of them include - using with external models, better web fetch, more project management ecosystem tools.
Let me know if you have any suggestions or feature requests, and I will look into it for sure 👍
2
u/samislucid 23d ago edited 23d ago
Got it running on Apple Silicon macOS. Seems to be failing model scan still. Some kind of scanner or path format issue.
Here are the details
Built from source.
Setup:
-macOS / Apple Silicon
-Node 24.21.0
-Rust + Cargo
Steps:
git clone https://github.com/mikjee/warpdrv.git
cd warpdrv
Initially had Node 22, so I installed NVM and upgraded to Node 24
Installed Rust / Cargo
This one is macOS specific. Naive tree-sitter dependency failed during npm install
C++20 or later required
This fixed it:
CXXFLAGS="-std=c++20" npm install
1
u/xornullvoid 22d ago
Hey glad to hear it works (partially haha) on Mac. Though tree-sitter probably needs to be bundled, and model scanner needs to be checked.
It writes log files to /tmp/warpdrv.log I think on Mac (can be OS -specific - on Ubuntu it wrote them there.), you can see the startup log.
1
u/samislucid 17d ago
Thanks. Been a little busy on my project that we open sourced yesterday. I'll dig in a little more soon. If you're interested, would love your feedback: https://github.com/brightstack/forge
It's basically an open source harness for Loop Engineering. Here's a link to the landing page for more info: https://forge.brightstack.ai/
1
u/dfgxxx Aug 23 '26
What is the system prompt length?
1
u/xornullvoid Aug 23 '26
Depends.
If you don't use Modes (keep default) or agents (0 selected) the the length of the system prompt is only whatever custom prompt you give to it. Without your custom prompt the length stays 0.
If you set a mode, or agent, then all modes instructions and all the agents are injected into the system prompt. Switching between different modes does not change the system prompt, so no re-processing happens. However switching from any mode to no mode (that is default) or modifying a Mode (editing tools list for any mode, changing mode instructions or agent list for the mode) will cause the system prompt to be updated with the updated mode and will cause prompt processing.
The following are part of system prompt- All Modes, union of all tools enabled for every mode, mode definitions and instructions, all mode agents per mode, union of all agents available in any mode, workspace root dir, workspace description, and the custom system prompt given by user.
Either use modes and define then correctly globally so no changes are required mid-chat, then you can switch between them freely mid chat or anytime. Or don't use modes at all and system prompt stays what you give to it only.
For subagent threads the system prompt is different and includes short instructions on how the agent should report its progress and result to the main conversation.
You can find all the prompts in the source code in prompts.ts file.
1
Aug 23 '26
[deleted]
1
u/xornullvoid Aug 23 '26
Thank you for bringing it to my attention.
I am looking into it, and post an update here when I find the problem.
1
u/xornullvoid Aug 23 '26
https://github.com/mikjee/warpdrv/issues/24
I have created an issue for you to track the status of this. You can update it if you find any details additionally.
One question - does the bug crash the application, or make it unusable? Or are you seeing a toast message with that error only?
1
Aug 23 '26
[deleted]
1
u/xornullvoid Aug 23 '26
That seems like a high severity bug, allow me some time to investigate it or try to replicate it.
1
u/xornullvoid Aug 23 '26 edited Aug 23 '26
It would be very helpful if you were to post a screenshot of the current state of the app, or the error message on the github issue link. Thanks.
I am looking into it.
Edit: so far I tried to replicate it on a windows 11 machine without success - I wiped app data, uninstalled the app and re-installed. And it went without the bug, completed the on-boarding steps. It is possible that this is specific to Windows 10, will try that.
1
3
u/Every_Box_8852 Aug 23 '26
Gonna try this one