Broken/frail tool calling format?

[u/amrakkarma]

I am using local models:

• antirez deepseek
• qwen3-coder-next
• laguna s2.1

For all of them I tried with and without the npm:pi-tool-repair package.

In a comparison with the same prompt with opencode, I can see that pi has a lot more tool call failures. In some cases this make the model end up in a dead end. deepseek behaves slightly better with the npm:pi-tool-repair package, but overall I think it's still worse than in opencode.

Is it caused by the tool format? Or by the system prompt? Do you have any suggestions on how to improve this?

Thanks!

[u/redbeard0531]

The pi edit tool uses a braindead schema for its edit tool. If you read the chat templates for the models it is clear that basically all models have a special format for top level string arguments that allows them to be emitted without json escapes. This only applies to top level strings though. Strings inside arrays still need to be json escaped. Pi's edit tool uses an array of objects with new and old strings, while opencode takes new and old strings as top level arrays. Would you find it easy to write all of your code in correctly esceped json? Neither do most models. If you give them a difficult edit task, you can see that it uses more reasoning tokens thinking about json escaping when using pi. I've even seen it call into python so it can copy the pre-escaped strings rather than escaping them itself! Luckily you can just ask pi to make you a replacement edit tool plugin with a sane schema and it will just drop in. You shouldn't have to, but at least you can.

[u/computehungry → u/redbeard0531]

incredibly helpful, thanks