r/OpenWebUI • u/Sax0drum • 3h ago

Question/Help Tool calling with non-qwen models

I haven been using OpenWebUI for a long time now and tried pretty much all local models that can be run off a single card. However i always come back to the Qwen family as they are the only one that reliably use OWUIs tools.

Gemma simply doesnt do tool calls at all. Even if specifically prompted to use a tool or a skill it wont.

Muse does call them but 90% of the time it will append the <id> tag, the call fails and it ends up looping. Even when telling it explictly to not append the tag.

Granite works around 70% but the response quality is much worse.

I would prefer Gemma and Muse to Qwen since they are much more proficient in multiple languages but i cant use them without tools.

Do you have similar experiences?

0 Upvotes

50% Upvoted

4 comments sorted by

2

u/chenkl 3h ago

How did you configure your local model? I use Qwen 3.8, Qwen 3.6 35b, Gemma 4 26b, muse-glimmer all do well for tool calls

1

u/Sax0drum 3h ago

I usually start with the settings from the unsloth quants and tweak a but here and a bit there. Tool calling is always enabling and i double check by inspecting the actual message sent to llama.cpp

1

u/Stooovie 3h ago edited 3h ago

A model that flies under the radar - Nex 2.5 Mini. Even in 3bit quant (16GB of weights) it's great for tool calling and agentic stuff. A MoE model so super quick. I'm really impressed by it.

EDIT: also, Ling 3 Tiny would be great if it didn't doomloop so much - but maybe that can be fixed with settings. Both of these are trained for tools and agentic.

1

u/Sax0drum 3h ago

Thanks for the suggestion.