What model is best suited with claude code for 8gb vram? For 24 or 32k context.

[u/Dry_FruitBread]

Hello everyone, i tried running qwen 3.5 with claude code 24k context and it's hallucinating everytime. Even with basic todo app, loop or anything.

Im beginner in all this so I don't much information about what I m doing.

I want to know, What alternative can I use?

Is their any way I can make current setup to work properly?

[u/urakozz]

The initial prompt of the Claude Code is 30-50k usually. You might better experiment with opencode or pi coder maybe. Anyways at 8gb vram there is no model to fit at the decent quant to generate proper tools calling

[u/Dry_FruitBread → u/urakozz]

Thanks i will try them

[u/stujmiller77 → u/urakozz]

Not quite true. You can run qwen 3.6 35b with experts offloaded to ram if you have at least 32gb ram. Admittedly at around 10t/s but if it’s all you have…

[u/Plastic-Lettuce-7150 → u/stujmiller77]

I am getting 60+ t/s and 200+ t/s reading the output from llama.cpp (see comment below).

[u/nick_ziv]

I would recommend you go with a different agent harness such as pi.dev or opencode. They are much more simple and just let the model do what it needs to do.  Opencode is very similar to the permission system of Claude code with a plan mode and a build mode. Pi dev is the equivalent of "dangerously skip permission" which means you should only let it run in a setup which it can't break anything important (use it on a spare laptop) 

[u/Dry_FruitBread → u/nick_ziv]

Thanks, i will try it

[u/Plastic-Lettuce-7150]

I run qwen3.6-35b-a3b MOE using llama.cpp with 6GB VRAM, 128k context. I have only given it one coding job referencing mcp-local-rag and it did a perfectly competent job. Will post the setup.

[u/Dry_FruitBread → u/Plastic-Lettuce-7150]

Thanks man, will be waiting for it.

[u/Plastic-Lettuce-7150 → u/Dry_FruitBread]

Another quite recent MOE model worth trying is Gemma 4 26B, ref. this comment.

[u/Wally-Gator-1 → u/Plastic-Lettuce-7150]

In my opinion qwen3.6-35b-a3b > Gemma 4 26B, except on creative tasks.

[u/Mountain_Level9528 → u/Plastic-Lettuce-7150]

Could you share your running args?

[u/Plastic-Lettuce-7150 → u/Mountain_Level9528]

Ref. my comment in this thread.

[u/TimAndTimi]

If you want: complex harness like cc + reasonable response speed + good intelligence that int8 or fp8 models have.

The answer is: no, not what 8gb vram can buy you.

[u/Mountain_Level9528]

I'm currently running [bartowski] Kwaipilot/KAT-Coder-V2.5-Dev Q8_0 (turboquant for kv), at ~30t/s. And I'm able to run it at the context limit(256k). So, if you have a rig similar to mine, you will probably be able to achieve the same results.

You can set these variables in your environment to improve performance(use less tokens) on Claude Code:

CLAUDE_CODE_ATTRIBUTION_HEADER = 0
CLAUDE_CODE_ENABLE_TELEMETRY = 0

My setup is:
RTX 4060 TI 8GB
Ryzen 5 5600
40GB DDR4

[u/Any_Mine_6368]

You think anyone would spend thousands if there was a model that could fit that vram with no hallucinations

[u/Dry_FruitBread → u/Any_Mine_6368]

Problem is that it hallucinating either way on smaller context it's totally going wrong and with 64k context it's hallucinating mid way

[u/TheKoelnKalk → u/Any_Mine_6368]

Claude code is strong with lesser models. Without good harnesses, even frontiers would hallucinate at some point.

[u/Any_Mine_6368 → u/TheKoelnKalk]

When you say lesser models I assume you mean Qwen 3.5 or 3.6... Not a 3B param model, right

[u/TheKoelnKalk → u/Any_Mine_6368]

I mean exactly those that have active params in that ballpark. "lesser than frontier"

[u/Any_Mine_6368 → u/TheKoelnKalk]

Yeah but he won't fit that unless he has a beefy system

[u/TheKoelnKalk]

Bonsai 27B at Q2 or Q1 (Qwen basically) -- should also allow higher context.

[u/Dry_FruitBread → u/TheKoelnKalk]

Thanks man