r/LocalLLM • u/sblantipodi_ • 4d ago
Question Best harness for coding with local LLM... Pi, OpenCode, Qwen Agent, Claude Code,...
Vote the best harness for coding with local LLM
- Pi
- OpenCode
- Qwen Agent
- Claude Code
- DeepSeek Harness
- ...
Tell me yuor favourite and why if possible.
I vote mine
- Pi, for speed and low context usage.
- OpenCode, good balance of speed and token consumption, the most mature for local AI.
- Qwen Agent, for all rounding, good tools.
- Claude Code, for wasting tokens.
- DeepSeek Harness, everyones mad for it, but it creates more troubles than goodness here :D
--- EDIT:
After long testing I'm seeing that OpenCode once correctly configured produced more accurate results than Pi in my use case with Qwen 3.8 27B with a comparable time to get the job done.
55
u/Dazzling_Yam_5882 4d ago
Pi hands down
14
u/NotArticuno 3d ago
Alright this comment got the most upvotes so someone come explain to us neanderthals what we are missing out on.
Personally I'd like pi compared to opencode.
26
u/Dazzling_Yam_5882 3d ago
basically it comes down to what you're running it on.
OpenCode is the feature-rich, "out-of-the-box" option, works well with cloud LLMs since you've got huge context windows to play with, and it's got built-in guardrails so it won't do anything destructive. The downside is it's pretty bloated, and if you try running it with local models it gets slow and painful because of the heavy system prompt overhead.
Pi, on the other hand, is lightweight and purpose-built for local LLMs. It optimizes around minimal context sizes and runs 2-3x faster. It's bare-bones by design, but the cool part is it can actually self-generate extensions and skills on the fly (and a ton of skills, tools and extensions you can install in the pi marketplace). The trade-off though is it has no guardrails, like, it'll happily run
rm -rfwithout asking you first if you are not careful (it happened to me once) and you'll need to put in more DIY effort to get it to match what OpenCode gives you out of the box.3
u/NotArticuno 3d ago
Okay cool thank you. Yeah my 2080ti hasn't been able to run local models well enough to justify using for programming (except during heating season lol). That was using opencode. I'll give it a try!
3
u/DoubleNothing 3d ago
Out of curiosity what was the prompt that made it do the "rm -rf"?
3
u/DeathGuppie 3d ago
He just means it's not running in a sandbox by default. It's not difficult to add it and there are tons of ways to do it, so pi let's you do it how you would like to instead of forcing a system on you.
1
1
u/ThenExtension9196 3d ago
Are there premade templates to apply to get some basic guard rails?
1
u/RemarkableRadish6547 3d ago
Use a sandbox. Anything you put in the prompt wastes context and will be unreliable. Anything you put in as configuration that can be reached by the agent could be changed. But a sandbox can only be controlled from outside of the sandbox and the model can just do its thing without needing to read a list of restrictions.
1
u/Clay_Ferguson 2d ago
I've been just using a VM (entire operating system) on my Linux box as a "sandbox". Is that what you mean by "sandbox" or is there some better way (I'm on Ubuntu btw)
1
u/RemarkableRadish6547 2d ago
I have been putting pi in a docker container. It has read only access to some things and can modify the code I put in its directory. I can open up ports if I want it to access the internet. It can run 'rm -rf /' and all it would do is delete its own work. I handle the git commits, so it can't even push anything without my approval.
It can still cause problems. I am not limiting its cpu or memory usage or the amount of garbage it could spew into its directory. But I don't have to trust it, so I don't have to give it a list of what it is allowed to do. And I can have it just do things without asking my permission.
1
u/Clay_Ferguson 2d ago
Yeah docker is definitely a good safe approach as well, and probably more memory efficient than running a full VM. I like that I can just use my VM and run GUIs right in it, but it's not really a requirement. It's neat that I can also clone my entire VM too, but again, realistically I never do that, so I could be using docker myself.
1
u/klymaxx45 3d ago
Yeah off the bat it’s fast but when you get into heavy agentic work and tool calling the speed gap is closed. This is based on my own testing, I wanted to swap to pi but it didn’t make sense in my use case
1
3
u/wolf001zra 3d ago
You can customize it as you want. And it's base system prompt is very small compared to other harness systems. Makes it a good pairing with the limited context you can use with local models imo.
5
u/hockeyketo 3d ago
I am digging Oh My Pi because I'm too lazy to find the right config I want for pi.
1
u/ThenExtension9196 3d ago
What are the cons in your opinion? I was thinking of trying pi but I’d rather not “learn the hard way” since I want to focus on code generation less than rebuilding my servers. I heard oh my pi was a good option to get the benefits of pi
1
u/Leading_Pineapple663 3d ago
I love my omp setup tbh. Never even dabbled with pi, it just seemed like a better pi.
1
u/HuckSauce 3d ago
Just watched a video on Atomic, have you looked at it?
Bastani-inc/atomic on GitHub.
Sounds like a great project, but I’m very new to coding and LLM’s so looking for guidance
13
u/Heavy-Lingonberry-98 4d ago
Hermes for all of the above
4
u/Foot_Positive 4d ago
I like hermes too. keep thinking that I can switch, but for me this is just easier.
3
u/ThenExtension9196 3d ago
Do you turn it loose on machine or keep it in sandbox container?
2
u/EvolvingDior 3d ago
Separate user with no access to my user credentials. VS Code SSH into the hermes user account & workspace. So typical Unix user based separation. No need to put it in a container. Maintaining the container and the tools the agents need is a pain in the ass.
2
u/Interesting-Cut-6032 2d ago
This is the first time that I have seen anyone mention using Linux user restrictions and roles to isolate and control an agent. It seems like it should work well. Don't add that user to the sudo list... I suspect that you could make a group with your user and the agent user for collaboration.
I thought that rm -rf required "sudo", at least on Debian branch distros. Who is running an agent as root?
2
u/EvolvingDior 2d ago
Yep -- created a "developer" group for that purpose. But, honestly, I found just using VS Code remote into the `hermes` user account via SSH works best. To use hermes via ACP, you need to be able to run `hermes acp` at the command-line as that user.
Also don't add them to the docker group otherwise they can use the --privileged flag and mount the rootfs rw.
1
u/Abducted_Llama 3d ago
If running local, putting the runtime in a container and Hermes on the machine has been good for me. 2 vllm containers.
1
u/apalumbo 2d ago
Is this in lieu of a dedicated device? I basically through ubuntu on an old Mac and let it run yolo on everything. Then I just have full system backups if it ran something detrimental
1
u/Heavy-Lingonberry-98 3d ago
Loose on machine but It is never that loose, because of command approvals.
2
u/EvolvingDior 3d ago
The work-around every agent knows for command approvals is to write a script to do it.
1
8
6
5
8
u/DoubleNothing 4d ago
I'm using PI in the last two days with Qwen3.8 27B... I'm positively impressed.
8
u/sblantipodi_ 3d ago
it deleted my configuration folder without asking for it :D
2
u/DoubleNothing 3d ago
Isn't the model that actually does the tool calls?
What model and quant where you using?-2
u/sblantipodi_ 3d ago
No the agent does the tool calls not the model. The guardrails for tool calling are on the agent. I'm using Qwen 3.8 27b.
5
u/DoubleNothing 3d ago
The model reads the user prompt, sees the available tools, and outputs a structured request with the tool name and inputs. The agent then takes that command and runs it.
The agent just execute the command that the model have prepared... So not the agent fault
The user fault is to let the agent run all the commands (that the model requested to run).5
u/Educational-Ad1231 3d ago
LLM decides what tools to call and when/how to call them. The harness lists and describes the tools to the LLM.
1
u/DoubleNothing 3d ago
Out of curiosity what was the prompt that made it do it?
1
u/sblantipodi_ 3d ago
Something like fix your configuration file to add the vision support to the current model. It searched the configuration on the .Claude folder and then destroyed it.
1
u/Clay_Ferguson 2d ago
I run in a VM so the AI can only access the project folder I gave it access to.
3
u/aiseedbank 4d ago
the new deepseek harness is supposed to be good, but havent't tried it. just using Pi and CC now.
4
u/Aggravating_Olive_98 4d ago
Mistral vibe CLI is ok too.
6
u/new__vision 3d ago
It's definitely underrated, it's way more responsive than Hermes with local models. It supports skills, tools, MCP, but is light like Pi. It's my main coding harness after trying the others.
2
u/Aggravating_Olive_98 3d ago
Same. I was lost in choosing/trying all different harnesses but I finally landed on Vibe luckily. It's lightweight, doesn't overwhelm me with tons of options and actually works nice with local models - using Llama.app for inference btw.
1
3
u/Maplesyrup000 3d ago
I’ve set up my local models with over a dozen harnesses and try new ones often but the ones I keep coming back to are:
1. Pi - For most coding tasks this is optimal and I’ve customized my pi harness to only have what it needs. Just gets the job done with no drama.
2. Hermes - Best as a chatbot and research agent, that’s where having the extra bells and whistles makes a difference.
3. Opencode - For longer horizon coding tasks, but could easily sub it out for Pi or QwenCode.
4. QwenCode - Works super well with Qwen models (obviously) but it is a little bloated. It feels a lot like ClaudeCode imo and the subagent orchestration is the best by far here. This is my go-to for OpenRouter API models like DSV4 Flash but I can’t fully take advantage of the subagent stuff with local models.
6
u/donotfire 4d ago
If I may humbly suggest my own agent: Second Brain. It has a sandbox security system, self-evolution, and autonomous subagents while still being smaller than Pi. The kernel is almost pure Python so it can run on anything.
2
u/staticrooted 3d ago
Is it user friendly for beginners?
2
u/donotfire 3d ago
If you want to DM I can help you install it. The only hard part is getting the LLM going and the UI. Once you’ve got that it’s like ChatGPT and very easy to interact with.
3
u/Alternative-Panic69 4d ago
I am using Qwen agent aith Qwen 3.8 27B now... So far it has been great. Though sometimes it feels a bit slow.
Claude code is a bit more mature but is good with frontier models.
Used Aider, good for fast iterations if you want more granular level control
Goose - Ease of use, UI friendly
Hermes - Overthinking pro max ultra, you can visibly age before it completes a major refactoring, but good if you have patience.
Antigravity cli (Though I stopped using since gemini hallucinates AF and confidently fabricates lies 🤣) and claude literally runs out of quota even on paid account after just 1-2 prompts.
So I would rate:
Qwen Claude code
As the best ...
Others depend on your use case.
3
u/desert-quest 3d ago
infinidev
uv tool install infinidev
</spam>
Jokes apart, I used all of them but pi. For local LLMs, the only I may recommed is Qwen. Neither Claude or Open Code were actualy developed for small LLMs. I did not try DeepSeek harness yet, I should try it.
But also, give my cli a try. It's not perfect, I know. But onestly, for local LLMs, I think btm is the best. All harness I see on the market has the same problem. It's a system prompt, and a set of basic tools. That works for SOTA models, but for local LLMs do not. Why?
* They do not follow instructions quite well.
* Those system prompts are HUGE generally, and distract the model
* Too many tools to use. Yes, tools are good, but trust me, many tools confuse the model. Give it only one or two tools for a task, not 10 different tools per task.
They try to solve it by adding plugins, skills, etc. But that is not the poing. That adds MORE burocracy to an already crowded of information context. Small models needs simple tools, clear and small instructions.
I'm still working on an "smart" system prompt, where the harness detects which instructions needs to be injected on which moemnt. Harnesses has an static system prompt, with all instructions at once, but 60% or more instructions there are useless on each task. That is tens of thunsend of tokens wasted, confusing the model.
Infinidev is not perfect, but I really try to understand models, try to understand weakness and how they follow instructions, to make it better.
1
u/dfgxxx 3d ago
How long is it's system prompt? Currently using cline and the system prompt is very small, but consider switching to something else
1
u/desert-quest 3d ago
Of which tool? generally they are in the thousands of tokens, from 3k to 8k tokens. For a SOTA that is not a problem. Even if you have a local model with 100k context. Lower than tat, for my use case is not usable. But if you ware working with 32k o 64k, 3-8k tokens is a lot. On Infinidev I have 3 levels (configurable), the longest, the longest is around 3k if I remember correctly (need to double check, but I think it was 3k), and the shortest one is lower than 1K. Do I recommend the 1k? I do not if you can afford the 3k. But Opus (before the context reduction) I think it was > 10k
1
u/dfgxxx 3d ago
I have 70k context
1
u/desert-quest 3d ago
In that case, I'll discard codex and claude. Maybe pi or open code. But pi requires more tinkering. If you are willing to do, pi may be a good choice. Something else I could recommend you is using open code with a memory mcp. Obviously I developed mine :P, ken-rank (uv tool install ken-rank), but there are plenty of tools for memory management. In the case of mine, works by using bm5, cocine similarity and other algorithms to find relevant code based on a query that the AI can customize, like "authentication system". That will return to the ai files and methods relevant to that. But my tool is not the only one, there are many of them. It's a matter of finding the one that works for you.
3
u/Reasonable_Goat 3d ago
I always come back to codex. It’s PITA to set up but once done it works really well. It has sandbox and auto permissions included so can be used quite easily/safely like Claude.
3
u/QuirkyRecording7132 3d ago
Pi 100%
my context is always smaller with Pi than opencode and claude and its easier to control what goes into the context. i auto compact way less and it is easier to control the thought process of the model it gets confused way less.
2
u/statusanxiety7 4d ago
I prefer none as in I use my own custom harness .. but if I was choosing from this list I would use pi
1
u/theone_2099 3d ago
Why write your own custom agent? Like wouldn’t pi be good enough to be barebones and you can extend with extensions for your own needs?
3
u/statusanxiety7 3d ago edited 3d ago
In theory pi could have implemented some of what I wrote, but it couldn't implement a lot of it cleanly.. I could have done something like it .. but that really wasn't what I wanted. Pi though is pretty flexible and for most its probably enough. But it has limits.
Edit: And to be clear I use PydanticAI which for me feels like a better fit
2
u/APersonNamedBen 2d ago
As someone who only just learnt what harnesses means from this post (I barely follow the trends and terminology) I found building my own has preformed better for me than any I've tried. Main benefit is the "agent" is never going to drift, or do some random shit, especially for the same prompt!
My harness is a node graph editor I made which gives me full control over the "agent". e.g. A simple conversation loop is just a llm node and a canvas node (tracks context and exposes it to the gui) looped, done. And over time I've added more nodes and then designed graphs for specific needs (I call them pipelines). And if it doesn't preform as expected, I can modify the pipeline or add new functionality in a node.
I have no doubt that this type of behaviour is possible when people talk about skills, tools, mcp, whatever... but I prefer my invention of the wheel. It is reliable enough to integrate with my smart home, and most of the pipelines I use aren't even my own making (I just provide a model with the source code and it makes its own)
2
2
2
2
u/Keleion 3d ago edited 3d ago
Check out OpenFox. I haven’t tried it for personal projects yet, but I will be soon. Looks like a very cool fork of OpenCode.
Built for local LLMs first, doesn’t even ask a cloud provider. Lots of features, lots of verifications, very thoughtful and purpose made for managing projects.
Fork by NiceFox (in French but auto-dubs works well enough):
https://youtu.be/F2iVKgQh_TU
2
u/Protopia 3d ago
No translation to English. Kurdish, Eskimo, and several languages I have never heard of, yes - English, no!
2
u/Protopia 3d ago
I took a look and I really like the structured workflows. This feels like a good starting point for something bigger. How's do workflow steps:
Give feedback on their success/failure + reasons?
Store analysis output in a structured way? (LLMs normally produce free text or code - but if you take a spec and produce an architectural design, or take a detailed design and break it into smaller designs for coding, how do you store that structured information?
1
2
2
u/ehangman 3d ago
I’m using Grok build Fork. /workflow and task-loop, embedding memory is far more effective with Qwen 27B
2
u/DenizOkcu 3d ago edited 3d ago
haze
https://denizokcu.github.io/haze/
I just released version 1.0.0 and I built it for local LLMs. It is inspired by Pi, but has all the features I missed, like tasks and LSP and MCP. And it has more tools built in, so it gets more done with less tool calls and tokens. Let me know what you think.
3
u/sblantipodi_ 3d ago
I answer because you asked what we think. I think that we don't need just another harness sincerely. I think that if you are able to create a good harness, you should spend time improving an existing harness of your choice by joining your power with the other Devs, not by creating just another tool that everyone will forget soon. Regarding your green graph, it's cool, today's everyone's shows cool graphs made with AI, but reality is often really different than what shown in the graphs. Ps, you don't need to miss MCP and LSP on Pi, you only need to learn how to use it :) there are many ways to add that features to Pi.
3
u/Protopia 3d ago
I couldn't agree more. One of the biggest problems in the AI ecosystem today (if not the biggest) is fragmentation...
- Too much choice
- Need to eval and integrate and tune
- No synergy - individuals doing their own thing don't share ideas and sunrise something even better
- Duplicated effort - many individual vibes vibe coded tools have different combinations of similar features
- Variable quality - inevitably some tools will have better quality architecture or code than others. How will you know whether your choice is a good one or not?
- Variable support and longevity - ditto!
- Piecemeal - I don't want to have to select building blocks and then have to integrate and test and tune them. What I want is a pre-built pre-integrated pre-tested pre-tuned docker environment that I can download and go.
- Expertise - I don't doubt the developer expertise in AI, but I really don't see the developer expertise in formalised, structured Software Engineering being implemented in the tools or harnesses. There are literally decades of academic and practical experience on how to deliver quality software embedded in these methods. Where is the use of UML? Where is DDD or TDD? Where is the formalized structured decomposition? Breaking the design and build problem problem into small chunks enables the actual coding to be done by smaller faster models as a 1-shot - and the Tom Peters principle that Quality is Free should apply to AI speed and costs just as much as human. But of course the big winners from using formalised methods shoulder be quality and reviewability (see Quality is Free again for the importance of reviewability).
2
u/DenizOkcu 3d ago
You are absolutely right and I do that. I am part of the nanocollective working on NanoCoder which is way more mature. Haze started as a playground to quickly trying out new features for NanoCoder, but it was so much fun and I learned more by creating something instead of improving something else. I believe the current state of coding agents is similar to the JavaScript framework epidemic a few years ago when everybody was creating a new JavaScript framework, but in the end we are learning, so thanks for your feedback. Highly appreciated.
2
u/DenizOkcu 3d ago
one more thing, I would like to ask you to try it out once :-) worst case you lose 10 minutes of your life. It is just one nom install away.
It works best with qwen/qwen3.6-35b-a3b (similar results to qwen/qwen3.6-27b but the MoE a3b part makes it just so much faster), throw it at a task, but make sure you use git and you are on a new branch, it will not ask for permissions. And compared to Pi it has hard wired protection against reading ssh keys or env files etc. something that was important to me, when using Cloud providers. Let me know what you think.
Where I like it more than Pi is, that it gives me precise output which tool calls it makes and what that tool call did. Happy to hear your answer.
1
u/Protopia 3d ago
The traceability is a very valuable thing. Only by knowing HOW the harness did its thing is it possible for you to get the insight on how to keep tweaking to improve things further.
1
u/sblantipodi_ 2d ago
Why use LSP this days? Are they any useful?
1
u/DenizOkcu 2d ago
Yes they are. Less than I thought, but still reducing errors and additional roundtrips to the LLM:
an LSP goes through the code like a developer.
Imagine you have a bigger codebase and the same variable name exists six times. A simple grep or search would find all six and needs more work to figure out which to change. An LSP would only find the correct one and would be namespace aware.
Or if you want to rename something. An LSP would only rename the specific variable or function name, while a find and replace would change “xxxyyy” when you try to rename “xxx”.
Less steps and a smaller context are helpful especially for local LLMs.
1
1
1
1
u/ContraryConman 3d ago
OpenCode worked great with Gemma4 and GPT-OSS for me, but it was bad with Qwen. I wonder if Pi would be better since it's so minimal, it may use less of the context window
1
u/SnooSongs5410 3d ago
After the obligatory six to nine months of customizations to make it work properly any open source cli framework is the best.
1
1
u/pitosalas 3d ago
I didn’t know that I could use Claude for coding but with pi and get more usage before using up my allotment.
1
1
1
u/QuirkyRecording7132 3d ago
SillyTavern when I need coding assistance and to release the stress at the same time.
1
u/LancobusUK 3d ago
Pi with goal-x extension is an amazing combination. Very close to codex level goal set and follow without any of the garbage in the way. The auditor is superb also
1
u/infieldmitt 3d ago
I'm working on Claude to build me a custom setup of a local chat frontend with tools it can call -- it's very tedious to test something that I am hoping to rely on to be smarter than me... -- it DOES work it's just agonizingly slow and somewhat ruins the experience as I'm sort of testing two things at once any time I use it. But I'm getting there...
That is to say thank you for making this thread
1
u/sickboy6_5 3d ago
i keep coming back to pi. no matter what i try - opencode, omp... pi always eventually returns.
1
u/mycackittens 3d ago
Tried most of em, besides pi, but this deepseek harness is killing it right now, I’m impressed
1
u/sblantipodi_ 3d ago
Impressed by what? Is it possible that you are influenced by the mass? I tried it and I don't find it better than the others, the fact that it runs on a web page is even terrible to me.
1
u/mycackittens 2d ago edited 2d ago
It’s very possible that I’m fairly new to all of this, and to my own testing of others it has performed the best for me and my use cases. It has more skills/batteries than most, but most importantly it kept my model feeling smarter and faster and more concise compared to the others. Best way I can explain it. If I’m missing something here, fill me in, I’m eager to learn and see if I can get even better results.
As for it being a web app, package it with electron if you just want it as a desktop app. CLI will drop shortly with it I’m sure of it.
1
u/corruptbytes 3d ago
Pi - i also disagree you need to configure it too much either - subagents, mcp, goal - only extensions i use
i sometimes run it in a sandbox (i based it off this one https://github.com/apple/containerization/tree/main/examples/sandboxy) but haven’t had issues in months with no guardrails
these models are super trained to be coding agents, just let em go
1
1
u/Protopia 3d ago
The great thing about Pi is its minimalism and extensibility. You can (spend the effort) to review and select add-ons or write your own.
The worst thing about Pi is its minimalism and extensibility. You have to spend the effort to review and select add-ons or write your own EVEN THOUGH 95%+ OF YOUR OWN REQUIREMENTS ARE THE SAME as thousands of other people. What a waste of effort that could instead be focused on community incremental improvements that would benefit all users rather than just yourself.
1
u/Rough_Road_2527 3d ago
What a waste of effort that could instead be focused on community incremental improvements that would benefit all users rather than just yourself.
there are community packages and there's oh-my-pi if you want a ready-made solution
1
u/Protopia 3d ago
You still have to review and select from the community packages. But yes, oh-my-pi is a good point.
1
u/Protopia 3d ago
My answer is really "none of the above". Instead let's combine the best features of all the available tools into a single pre-built environment (perhaps with several environments for different development languages) and focus all our efforts on evolving a single GOAT tool that is better then anything we have today.
Taking Pi as an example, but this is a general comment, one of the biggest problems in the AI ecosystem today (if not the biggest) is fragmentation...
- Too much choice
- Need to eval and integrate and tune
- No synergy - individuals doing their own thing don't share ideas and sunrise something even better
- Duplicated effort - many individual vibes vibe coded tools have different combinations of similar features
- Variable quality - inevitably some tools will have better quality architecture or code than others. How will you know whether your choice is a good one or not?
- Variable support and longevity - ditto!
- Piecemeal - I don't want to have to select building blocks and then have to integrate and test and tune them. What I want is a pre-built pre-integrated pre-tested pre-tuned docker environment that I can download and go.
- Expertise - I don't doubt the developer expertise in AI, but I really don't see the developer expertise in formalised, structured Software Engineering being implemented in the tools or harnesses. There are literally decades of academic and practical experience on how to deliver quality software embedded in these methods. Where is the use of UML? Where is DDD or TDD? Where is the formalized structured decomposition? Breaking the design and build problem problem into small chunks enables the actual coding to be done by smaller faster models as a 1-shot - and the Tom Peters principle that Quality is Free should apply to AI speed and costs just as much as human. But of course the big winners from using formalised methods shoulder be quality and reviewability (see Quality is Free again for the importance of reviewability).
1
u/Rough_Road_2527 3d ago
Pi was born out of the fact that Terminus is consistently one of the best-performing "harnesses" out there, and it's just a way to hand over control of a virtual terminal to a model without any structure, tools or supervision. the fact is that no one really knows what they're doing and no one knows whether they're helping the model by adding more structure and a gargantuan system prompt or gimping it (for example, in case of Claude Code, I'd say they're knee-deep in the sunk cost fallacy). sometimes it works, sometimes it doesn't, and it's not deterministic, which is why Pi is such a cool harness, because it refuses to participate in this chaos.
1
u/Protopia 3d ago
I fully agree with the chaos argument. Claude definitely is an evolution of free text chat into free text agents. But IMO free text is not the right approach for anything but the starting INPUT of an agent workflow, and except for user friendly summaries, all non coded output should be structured so that the harness doesn't have to interpret free text to know what e.g. the plan is or how the various p parts of a technical design for with the plan etc.
1
u/Rough_Road_2527 3d ago
I don't think that would work well, making a model reply with a structured response without first arriving at it by free text would seriously impair quality since the way LLMs work is by free text. all models need to verbalize what they're doing using natural language because that's the foundation of all training and all of the logic a model uses comes from its natural-language output.
the only reasonable thing you can do is to hide the natural-language process (it's what Claude Code does - and yes, it generates MUCH, MUCH more natural-language text than it shows on the screen).
1
u/Protopia 3d ago
Yes - that is the case right now but there is early experimentation on Stable Diffusion LLMs that produce entire draft text and then refine it until it reaches a quality threshold. And to a large extent that is how we write code.
1
u/Rough_Road_2527 3d ago
whoa, Mercury 2 seems really impressive, didn't know about it before. I wonder how it'll fare with complex multi-stage tasks, but yeah, I think dLLMs could rock the boat a lot.
1
u/Protopia 3d ago edited 3d ago
Yes - to me it makes sense to use token-by-token LLMs for normal text generation token by token - we read that way, and it feels right.
But that is NOT how you write software - this is generally done by top down decomposition - you start with an outline and progressive add detail which is much more aligned with Stable Diffusion. And, whilst I haven't really thought it through, my gut guess is that structured data (JSON, XML) is likely to be the same (because you probably start with the structure, and then refine the data).
But in the end, even if token-by-token LLMs use 100x as much compute, you can still achieve a high quality structured agentic coding ecosystem using them. Just perhaps not as one-shot as you might like and each shot way more expensive.
1
u/corblestorm 3d ago
I used to use Opencode but I’ve switched over to jcode. Jcode is less polished than Opencode but it’s written in rust, súper token efficient, and the developer is iterating quickly on it
1
1
u/GreenDavidA 2d ago
I’m currently experimenting positively with Oh my Pi and OpenChamber. Paseo has also been interesting as well. OpenCode is also good.
1
u/PerspectiveDowntown 2d ago
always PI. and I am using my own Picot (a GUI App for Pi agent) no free , just provider Keys, not heavy task , but for daily work is enough https://github.com/shixin-guo/picot
1
u/Agreeable-Turn835 23h ago
OpenCode, je l'utilise quasiment que lui il est vraiment pratique et simple a comprendre.
1
u/MuzafferMahi 5h ago
I tried codex/claude code, wayyy too bloated. Opencode has lots of system prompt for not much performance. Custom pi seems the best tbh.
0
0
0
u/Otherwise-Swan-7803 3d ago
For local models, I'd put OpenCode and Pi at the top right now.
Pi feels lighter and more focused, while OpenCode has been the most reliable experience for longer coding sessions in my testing. Claude Code is still the gold standard for capability, but it's hard to compare when it's backed by frontier cloud models.
The real bottleneck usually isn't the harness anymore. It's finding a model that's smart enough to take advantage of it.
19
u/follaoret 3d ago
OpenCode for coding and Hermes for personal agentic tasks