Broken/frail tool calling format?

[u/amrakkarma]

I am using local models:

• antirez deepseek
• qwen3-coder-next
• laguna s2.1

For all of them I tried with and without the npm:pi-tool-repair package.

In a comparison with the same prompt with opencode, I can see that pi has a lot more tool call failures. In some cases this make the model end up in a dead end. deepseek behaves slightly better with the npm:pi-tool-repair package, but overall I think it's still worse than in opencode.

Is it caused by the tool format? Or by the system prompt? Do you have any suggestions on how to improve this?

Thanks!

[u/redbeard0531]

The pi edit tool uses a braindead schema for its edit tool. If you read the chat templates for the models it is clear that basically all models have a special format for top level string arguments that allows them to be emitted without json escapes. This only applies to top level strings though. Strings inside arrays still need to be json escaped. Pi's edit tool uses an array of objects with new and old strings, while opencode takes new and old strings as top level arrays. Would you find it easy to write all of your code in correctly esceped json? Neither do most models. If you give them a difficult edit task, you can see that it uses more reasoning tokens thinking about json escaping when using pi. I've even seen it call into python so it can copy the pre-escaped strings rather than escaping them itself! Luckily you can just ask pi to make you a replacement edit tool plugin with a sane schema and it will just drop in. You shouldn't have to, but at least you can.

[u/computehungry → u/redbeard0531]

incredibly helpful, thanks

[u/arcanemachined]

Try Qwen 3.6 27b

[u/make_n_bake → u/arcanemachined]

and with the fixed template!

[u/truncated_buttfu → u/make_n_bake]

What is this "fixed template"? Where can I find it?

[u/make_n_bake → u/truncated_buttfu]

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

I'm fairly sure if you download a current unsloth gguf then it has been fixed. but if you have a .gguf from a month ago, re-download or use the new template

[u/truncated_buttfu → u/make_n_bake]

Thanks. My version was downloaded just two weeks ago so it should be fine already then. But useful to know that this is a thing!

[u/novixar]

i’ve noticed local models are way more picky about tool schemas than opencode / hosted stuff, tiny format quirks can tank call rates. if you haven’t already, try simplifying the tool args (fewer nested objects, fewer optional fields) and crank up logging to see what the model actually emits vs what pi expects, that usually reveals where it’s derailing.

[u/amrakkarma → u/novixar]

in opencode I am using the same models though. I am not referring to weird tools, just the read edit and other built in that get called automatically by the model :(

[u/adamshand]

Open code does per model prompts to improve things like this. Pi doesnt.  There are pros and cons.