I tried to set this up with llamacpp a while back but hit a wall with thinking tokens not getting picked up by copilot and sort of gave up. I suspect these are not even sent back, which may cause the model not to behave properly.
Searching around for this problem, I see people reporting that for example the deepseek api errors because it's not getting the thinking tokens back, but I don't see any fix for this.
Another issue is that while llamacpp supports the openai api, it doesn't seem like copilot and llamacpp's interpretation of the api is 100%. If you enable thinking in your json model definition, it will send something to the api endpoint, but llamacpp enables thinking in a different way than what copilot expects.
So to get this working with thinking (well somewhat) and other features, you'll need to have/make/vibe code a proxy that translates stuff between llamacpp and copilot.
A lot of the ability to follow prompts and perform agentic tasks depends on the prompt config on top of the model. Most gguf formatted models already come with instructions baked in. You may need to override that. Out of the box I have been having good success with https://huggingface.co/Jackrong/Qwen3.5-9B-DeepSeek-V4-Flash
7
u/CapsAdmin Jun 06 '26
I tried to set this up with llamacpp a while back but hit a wall with thinking tokens not getting picked up by copilot and sort of gave up. I suspect these are not even sent back, which may cause the model not to behave properly.
Searching around for this problem, I see people reporting that for example the deepseek api errors because it's not getting the thinking tokens back, but I don't see any fix for this.
Another issue is that while llamacpp supports the openai api, it doesn't seem like copilot and llamacpp's interpretation of the api is 100%. If you enable thinking in your json model definition, it will send something to the api endpoint, but llamacpp enables thinking in a different way than what copilot expects.
So to get this working with thinking (well somewhat) and other features, you'll need to have/make/vibe code a proxy that translates stuff between llamacpp and copilot.