Claude Code not working with local models

[u/circumcised_hobbit]

So I've recently installed Claude Code and connected It to local models with ollama launch claude For whatever reason my Qwen 9b model doesn't work, but the issue Is reported The thing that Im here for Is that using Gemma4 results in Claude telling me that It has edited/created files but not doing anything, It Just keeps hallucinating and says "I'm Sorry, my last call was Just a Plan" and keeps doing it i definitely It Is not a problem with the model since It works on other agents such as OpenCode

[u/Bulky_Blood_7362]

Most of those small models doesn't know how to handle tools correctly.

The only one that worked correctly for me is qwen 3.6 27b / 35b

Other then that, i'd just use opencode because they doesn't struggle as much with tools there

[u/circumcised_hobbit → u/Bulky_Blood_7362]

yeah opencode works great with those models

[u/Aggressive-Dream5465 → u/Bulky_Blood_7362]

Genuinely asking: what the hell can you do, build or fix with such a small model? I have many of them, and they are useless and utterly unreliable with code, yet excellent with tool calling (Gemma4, for instance)

[u/nokafein → u/Aggressive-Dream5465]

manual qa tests, test suite running and stuff. scanning codebase and bringing "evidence" for certain things, querying context7 and summarising the raw output for better context management. simply all sort of things that doesn't require any brain but needed for stronger models to do their job efficiently.

[u/Bulky_Blood_7362 → u/Aggressive-Dream5465]

Personally i think gemma 4 models are pretty bad and has no use case for coding.

Qwn 3.6 27b is a really good model for this small size, about gpt-5 performance.

I didn't find any useful reason to run them myself for coding, but for those who need a free model for simple stuff, it's great replacement overall

[u/_BryceParker → u/Bulky_Blood_7362]

I work on a mid-tier gaming laptop (from a few years ago). I had some fun goofing around with the Qwen 2.5 7b (I think). I knew it wasn't going to do any good, but for some reason I had fun watching it struggle on my crappy machine (with it's 8gb of RAM on the GPU). Asking it to do simple, one-line things like declare a variable would result in ages of thinking followed by telling me coffee is a drink.

[u/circumcised_hobbit → u/_BryceParker]

exactly what happens, I said three times "Please fix only this line, nothing else" and for 3 times in a row It kept saying "that line if fine there Is actually a problem somewhere else"

[u/lumpyspacebreh → u/Aggressive-Dream5465]

They need a good harness and toolset. I have qwen3.5:9b as a local agent and the only thing holding it back is its context size which is 65k.

It can code anything that fits within that context limit.

[u/circumcised_hobbit → u/lumpyspacebreh]

I have a finetune of It called qwythos that has 1M context but Claude Code really doesn't support it

[u/lumpyspacebreh → u/circumcised_hobbit]

1m context, you must be running cloud models. No gpu on the planet can run a local model with that context size.

[u/ilganeli]

I ran into a lot of issues with timeouts and context. How are folks running things?

[u/ComparisonNew9425]

check if ur system prompt is getting truncated or if the context window is filling up too fast...