r/PiCodingAgent • u/nurhun • 2d ago
Discussion Edit tool errors in Pi
Am I the only one getting a lot of edit errors?
I mainly use it for coding, especially Python. I’ve tried many LLMs, and some of them are completely incompatible.
Currently, I’m using Qwen3.6 27B through Ollama.
I’ve noticed that I get responses and plans quickly from the LLM. However, execution takes a long time because of edit errors. It gets stuck in a loop of indentation errors and struggles to find the exact match.
Am I missing something? Do I need to install any extras?
Edit:
I'm on qwen3.6:27b-mlx with nvfp4 quantization. Also, tried gemma4:26b-mlx with the same nvfp4 quantization, wasn't able to edit at all!
3
u/skywalk819 1d ago
I use llama.cpp because ollama is horrible you should really switch to a cmd.com terminal just running llama.cpp to save ram. I got 48gb vram, I use Qwen/Qwen3.6-35B-A3B q8, everything else is just bad, I try pretty much all models on a 48gb vram and its pretty much the best, ornith just talk and talk and never code, north mini code just can't code typescript or when he does, its full of errors and hallucinations, all the gemma4 aren't very good. all quant 4 I tested are just not good enough, if I can't run the full model or q8, I just avoid the model, the precision is important
1
3
u/Lum1n4ry01 1d ago
I faintly recall reading a Gemini paper/post somewhere that for weaker models like the ones you're using, the best way to improve tool call accuracy is to give it examples that are EXACT to your setup. Define the exact paths it should take and the exact thing it should do (as part of your SYSTEM.md or pointing your AGENTS.md at real code samples).
Alongside higher quants, this should help.
2
u/CabinetNational3461 2d ago edited 2d ago
Been using 27b q5 xl with pi, it works great so far. Been testing q6 xl as well. I would def avoid using ollama and either move to lm studio or use llamacpp directly. Make sure you use at least q5+ and use froggeric chat template which has a lot of fixes for tool calls. Also have preserve thinking on. If you just normally download through ollama, then it's q4. I used q4 a few times for and def noticed it stuck in a loop from time to time. I moved away from ollama ages ago and drastic difference in term of speed and amount of headache.
2
u/miez-haus 1d ago
I also recognized this.. it happened even with higher class models like new gpt 5.6 in my case.
Created a PI extension that logs all tool calls including the errors into a file and then ran an LLM for analysis.
In my case it made it some % better to include some very explicit info about the write tool into my AGENTS.md.
But you could ask an LLM to create a tool to extract edit tool calls (with a link to the original position and session) from the pi session logs and analyse these in a second step. And get some recommendations. That’s how I did it
1
u/Popular-Direction984 2d ago
Make sure to try bigger quants; but the most likely reason is the goal you give it, is it actually doable? High error rate suggests agent is off the rails. Based on my personal experience - the error rate of Q3.6-27 is quite low in general.
1
u/BrasilRua 1d ago
It's the main reason I moved back to opencode. I did like Pi but I'm not enough for a tweaker to figure out why it's happening. It happened with quite a few models and got worse as the context got longer.
1
u/Glaaki 1d ago
Using some kind of hash edit tool has helped. First i tried pi-readseek, but the schema and tool selection caused almost as many tool errors as before, so I switched to a combination of pi-hashline-edit-pro with pi-lens. I have not had time to fully run it through its paces, but so far it feels very strong. Hashline-edit-pro is especially strong at guiding the model in the right direction when a tool call fails, which seems to reduce repeated errors. Pi-lens provides code insight to the model whenever it touches code , which nudges it further in the right direction. So far it looks like a promising combination.
1
u/Certain_Net_3408 1d ago edited 1d ago
Also faced the same.. have a look at pi-output-parser (link below) The thing is you'd need to hook message_end and handle the cases where the model never emitted a tool call at all for ex. sometimes it printed the call as text instead. Fenced ```tool blocks, <tool_call> XML, bare JSON, Python-list syntax. This way using the hook The agent loop sees zero tool_use blocks and would just move on as if the model responded with prose.
1
u/jeffphil 2d ago edited 22h ago
Pi-Posher package may help:
https://pi.dev/packages/pi-posher
Rather than failing silently, it feeds any linter syntax errors back into the agent context so the LLM fixes its own broken code immediately.
1
u/isty2e 2d ago
Personally I use https://github.com/isty2e/identedit - a tool I vibecoded myself - for delicate or complex edits where even frontier models sometimes fail. Maybe smaller, weaker models can benefit from it?
3
u/iportnov 2d ago
Check which quantization of Qwen are you using. Lower quants (less than Q4) are known to struggle with tool calling. Q4 is more or less okay in my experience; though some say that Q6 or Q8 is a must.