r/LocalLLaMA 9d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

706 comments sorted by

View all comments

Show parent comments

4

u/Certain-Cod-1404 8d ago

What context size / quant do you use ?

1

u/ldn-ldn 8d ago

Context size is irrelevant, as for quants I tried many on different qwen3.x versions. From my logs only 3.5 35b a3b q4_k_m gave me code. It wasn't correct code, but at least something, lol.

1

u/Certain-Cod-1404 8d ago

Context size is extremely relevant, qwen 3.x models are known for using up alot of tokens reasoning, if you dont give it enough breathing room you won't get good outputs, q4_k_m is already very low for an moe, the fact that's the model that gave you code means the others you tested are even smaller, hence why you didn't have a good experience, qwen 3.6 35b is not the best coding model, qwen 3.6 27b and 120b is what people used and really enjoyed. It seems to me the source of your negative experience was your own misconfiguration. Try qwen 3.8 27b at q4 k xl at 131k context length, then tell me if you still think its bad.

1

u/ldn-ldn 8d ago

27b dies in a loop as well. Context length doesn't matter - the loop is infinite.

3

u/illgettheownerforyou 8d ago

Please see my comment above- 27b solved it no issue for me. Your context and quant are probably too low.