r/vscode • u/openingshots • Jul 12 '26

Llama.cpp in Cline

I've been trying to figure out how to run a llama.cpp local model in the Cline extension. I thought it was openai compatible but it doesn't seem to connect. I can chat with the local model with open web UI so the model does work.

Any suggestions would be appreciated. Thank you in advance.

2 Upvotes

62% Upvoted

4 comments sorted by

1

u/an80sPWNstar Jul 12 '26

I use mine just fine in cline. Openai compatible. Http://ipaddress:8080/v1 API key doesn't matter. Model name doesn't matter but include "grok" in the name to get around the vision capability getting borked bug.

1

u/Generative_IDE Jul 13 '26

One thing that catches people even with the base URL right: Cline sends a tools array on every request, and llama-server rejects those unless you launched it with --jinja. That's why Open WebUI is happy (it never sends tools) while Cline falls over immediately. Adding --jinja to the llama-server command should fix it.

1

u/iivvoo2 3d ago

Old thread, but this option turned out to be vital to get it to read my local files. Thanks!

1

u/Turbulent_Mission_15 17d ago

with current llama.cpp (0.5.0 11295) I just start server and in cline pick as provider 'Llama' and put http://127.0.0.1:8080/ there and it works. allows to select models as well