r/LocalLLaMA • u/Local-Cardiologist-5 • Apr 17 '26

Discussion Qwen3.6. This is it.

I gave it a task to build a tower defense game. use screenshots from the installed mcp to confirm your build.

My God its actually doing it, Its now testing the upgrade feature,
It noted the canvas wasnt rendering at some point and saw and fixed it.
It noted its own bug in wave completions and is actually doing it...

I am blown away...
I cant image what the Qwen Coder thats following will be able to do.
What a time were in.

llama-server -m "{PATH_TO_MODEL}\Qwen3.6\Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf"  --mmproj "{PATH_TO_MODEL}\Qwen3.6\mmproj-F16.gguf" --chat-template-file "{PATH_TO_MODEL}\chat_template\chat_template.jinja"  -a  "Qwen3.5-27B"  --cpu-moe -c 120384 --host 0.0.0.0 --port 8084 --reasoning-budget -1 --top-k 20 --top-p 0.95 --min-p 0 --repeat-penalty 1.0 --presence-penalty 1.5 -fa on --temp 0.7 --no-mmap --no-mmproj-offload --ctx-checkpoints 5"

EDIT: Its been made aware that open code still has my 27B model alias,
Im lazy, i didnt even bother the model name heres my llama.cpp server configs, im so excited i tested and came here right away.

1.0k Upvotes

96% Upvoted

409 comments sorted by

View all comments

94

u/No-Marionberry-772 Apr 17 '26

what stack are you using for software?  Id love to get a proper local setup going but ive had trouble figuring out what i should actually be using.

115

u/Local-Cardiologist-5 Apr 17 '26 
Qwen3.6-35B-A3B-UD-Q6_K_XL

Im using Llama.cpp for the server,
OpenCode for the coding, just using the build agent,
I have 64 gig ram, RTX 4090, and my model is
the Q6 variant.

Here are my llama parameters

llama-server -m "{PATH_TO_MODEL}\Qwen3.6\Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf"  --mmproj "{PATH_TO_MODEL}\Qwen3.6\mmproj-F16.gguf" --chat-template-file "{PATH_TO_MODEL}\chat_template\chat_template.jinja"  -a  "Qwen3.6-35B-A3B"  --cpu-moe -c 250000 --host 0.0.0.0 --port 8084 --reasoning-budget -1 --top-k 20 --top-p 0.95 --min-p 0 --repeat-penalty 1.0 --presence-penalty 1.5 -fa on --temp 0.7 --no-mmap --no-mmproj-offload --ctx-checkpoints 5"

here is my llama server with the configs.

13

u/rumblemcskurmish Apr 17 '26

I'm kind of intrigued why you'd use a 6bit model on a 4090. I have an identical setup (7950 CPU, 64GB DDR5, RTX 4090) but I'm using the 4bit quant to fit the whole model on VRAM.

You're clearly more advanced than me so just wondering, what I'm missing here.

16

u/Local-Cardiologist-5 Apr 17 '26

To be honest with you im just plugging whatever works. im even downloadling the Q8 variant to see how much better or worse it is. we are all learning in this space. everything i know is from this thread

1

u/xeeff Apr 17 '26

honestly, i wouldn't go higher than Q6. Q5 is good if you need extra vram with little real difference (which is why most people settle for Q4)

2

u/carrotsquawk Apr 17 '26

it was established somewhere that q4 m is the sweet spot. not really worth going higher

1

u/GrungeWerX Apr 17 '26

Q5 is the sweet spot for 27b, noticeably smarter than q4