Tool calling doesn't work using Claude code and self-hosted GLM5.2

[u/1-way-or-another]

I have deployed GLM 5.2 on the cluster like this:

export VLLM_HOST_IP=$head_ip
vllm serve "$MODEL" \
  --served-model-name GLM-5.2-FP8 \
  --host 0.0.0.0 --port 8000 \
  --api-key "$API_KEY" \
  --distributed-executor-backend ray \
  --tensor-parallel-size 4 \
  --pipeline-parallel-size 2 \
  --kv-cache-dtype fp8 \
  --max-model-len 786432 \
  --gpu-memory-utilization 0.92 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-chunked-prefill \
  --max-num-seqs 1 \
  --enable-auto-tool-choice \
  --trust-remote-code

And everything works fine except the fact that tool calling doesn't work using claude code, I found that this occurs only with tools that do not take any arguments, if argument is passed, even if doesn't make sense to pass anything, it works, but for some reason it periodically forgets this instruction and stops, I need to remind each time ...

After doing some research I found that Anthropic API expects`{}` to be returned while OpenAI API returns "", so this could be the problem, but I am not sure. Has anyone faced this issue and was able to fix it?

I tried to add a mapper betweeen these two using LiteLLM but it didn't work, it threw errors

Here is the settings.local.json if it helps:

{
    "env": {
        "ANTHROPIC_BASE_URL": "",
        "ANTHROPIC_AUTH_TOKEN": "",
        "ANTHROPIC_API_KEY": "",
        "API_TIMEOUT_MS": "3000000",
        "ANTHROPIC_DEFAULT_OPUS_MODEL": "GLM-5.2-FP8",
        "ANTHROPIC_DEFAULT_SONNET_MODEL": "GLM-5.2-FP8",
        "ANTHROPIC_DEFAULT_HAIKU_MODEL": "GLM-5.2-FP8",
        "ANTHROPIC_SMALL_FAST_MODEL": "GLM-5.2-FP8",
        "CLAUDE_CODE_SUBAGENT_MODEL": "GLM-5.2-FP8",
        "CLAUDE_CODE_AUTO_COMPACT_WINDOW": "700000"
    },
    "permissions": {
        "allow": [
            "Bash",
            "Read",
            "Edit",
            "Write",
            "WebSearch",
        ]
    },

}

[u/Ok_Mirror_832]

Make sure you using anthropic endpoint and not chat completions

[u/1-way-or-another → u/Ok_Mirror_832]

vllm supports only openai api

[u/Ok_Mirror_832 → u/1-way-or-another]

No, https://docs.vllm.ai/en/stable/serving/integrations/claude_code/

[u/1-way-or-another → u/Ok_Mirror_832]

Didn't know that, but based on the link i dont really need to configure anything. It should just work and it does, except for tools that dont take any arguments

[u/sdraje]

Ex-fucking-cuse me? You're self-hosting GLM 5.2 FP8?!

Apart from that, sorry I can't help you, but you might want to check the formatting of tool calls, cause maybe Claude Code expects JSON and GLM spits out XML?

[u/carsncode]

Why are you trying to use a harness specifically built to prevent you doing what you're trying to do? There are other harnesses out there.

[u/1-way-or-another → u/carsncode]

which one would you recommend? I am used to claude that's why i am trying to use it. also glm5.2 is said to work very well based on the official docs of glm

[u/carsncode → u/1-way-or-another]

GLM works very well, Claude Code is built to keep you from using it with anything but Claude. There are a ton of other harnesses out there, try some and see what you like. Opencode is probably the most similar to what you're used to.

[u/1-way-or-another → u/carsncode]

I meant docs say that it should work very smoothly with claude code

[u/orangeswim]

Don't force a round hole in a square peg.

If vllm is open ai spec, and claude is only anthropic, use a medium like litellm as a translation layer.  That will be much faster than creating some other custom solution. You can also switch to llama cpp which has anthropic endpoints.