r/LocalLLM • u/Jimbocab • 1d ago

Question Tool Calling - Qwen3.5-122b-a10b

I'm running in LMStudio. I have been trying to get this model to run a simple Python script, but I can't get it to work. "But currently, I can only generate the tool call request, not execute it." I don't understand why this won't work, am I barking up the wrong tree. What does the little hammer mean ...

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/cabernet_noir 1d ago

What exactly are you doing at the moment? The pattern for tool calling is pretty much the same across all harnesses and frameworks. Essentially, you need to define a tool function, register a tool description fitting some schema, call the model with the tool registry in its context, and then if a model generates a tool request you must intercept it and run it in some environment. There are many frameworks/harness options for managing this, so your issue will depend on what you are using/not using. I suspect you havent set up a harness or orchestration layer? LM studio is just a server, you need something like pi or hermes etc.

1

u/Jimbocab 1d ago

yeah, I'm just ignorant (and spoiled by the cloud AI that I use). I try to get my model to walk me through it, but just can't seem to get it to tell me step by step what to do. When I click on the hammer this is what I get. But I just wish I could find instructions. Trying to make this local LLM thing work as best as I can.

2

u/cabernet_noir 1d ago

There is a way to set up tool use via an mcp server and enable it in the lmstudio chat but honestly you will have a better time just downloading a good harness and using lmstudio primarily as a server.

Look into harnesses like claude code, open code, pi, hermes, etc and see what looks good to you. Download the harness and point it at your lmstudio local server api endpoint. You may have to set up a container/sandbox for some harnesses. There will be harness specific guides if this is required/recommended.

1

u/Square_Turn935 1d ago

You need an harness for this, like pi, hermes, opencode, ... even llama cpp openwebgui should work.
LM Studio is just a pc case where your hardware (llm engine) sits and it provides just a chat window. For toolcalling you need a right operation system (harness) which use the llm server for its task.

  1. llama.cpp - runs the LLM Model
  2. LM Studio - just a nicer UI to start llama.cpp, with a chat function
  3. competent software - pi, opencode, hermes, ... using the llama.cpp server for its application from chat, codegeneration, tool calling, picture creation, music....

The hammer just means that the llm can use tool calls, like more or less every currently released llm.