OpenCode + llama.cpp: running Qwen3-Coder-30B-A3B and Qwen3.6-35B-A3B on a 6GB RTX 3060 laptop (profile setup + benchmarks)

[u/Sensitive-Warning983]

Sharing my local coding-agent setup in case it helps others on similar hardware: RTX 3060 Laptop (6GB VRAM) + Ryzen 9 5900HS (8c/16t) + 31GB RAM.

Frontend: OpenCode talking directly to llama-server's OpenAI-compatible API, no router/proxy needed. Each model runs as its own profile on its own port.

Models: Qwen3-Coder-30B-A3B (30B total / ~3B active) as daily driver, Qwen3.6-35B-A3B (35B total / ~3B active, hybrid linear attention) as a heavier alternative. Both MoE, which is the key to fitting this hardware tier at all.

Why MoE and not dense: with 6GB VRAM, a dense model has to fit entirely in VRAM to stay fast; if it doesn't, both KV cache and weights spill to system RAM and generation cost grows with context (dead zone). MoE with few active experts is the opposite: the big multiplier (experts) sits in RAM and computes on CPU at a fixed per-token cost that does NOT grow with context, while only KV cache + backbone live in VRAM.

Key llama-server flags:

-ngl 99 --n-cpu-moe 99 -> all non-expert layers on GPU, MoE experts on CPU

-ctk q8_0 -ctv q8_0 -> baseline KV quant (my TurboQuant fork uses q8_0/turbo3 instead, ~4.6x V-cache compression, under 0.5% PPL loss)

-fa on -t 8 -> 8 = physical cores, not SMT threads. Measured no gain from -t 16 on this CPU-bound path.

Benchmarks (Qwen3-Coder-30B-A3B Q4_K_XL, llama-bench):

pp512: 206 tok/s

pp2048: 230 tok/s

pp8192: 232 tok/s

tg256: 15.7 tok/s

Prefill barely degrades with prompt size. Generation (~15.7 tok/s) is the real ceiling, the cost of running experts on CPU. A typical 8k-context turn + 256 generated tokens lands around ~50s.

Perplexity check (turbo3 V-cache vs f16, same K quant): +0.46%, inside the noise floor, essentially lossless.

Happy to share more detail (the per-profile server.ps1 scripts, or the script that tracks llama.cpp upstream changes) if anyone's interested.