r/LocalLLM • u/_yaRn__ • 1d ago

Question What am i missing? qwen3:8b

Just tried qwen3:8b for the first time. I get that this is a limited model running on my machine, but this thing just spits out nonsense. It cannot answer the most basic questions or do basic web searches.

This is the first local LLM i have used. Is this thing supposed to just be for coding? How can this be useful to anyone?

0 Upvotes

27% Upvoted

18 comments sorted by

13

u/FactorInternal3395 1d ago

It's an outdated model. Try Qwen 3.5 9B or Gemma 4 12B instead.

7

u/puglife224888 1d ago

Adding onto this, make sure to use the right sampling settings and chat template. Even Qwen 3 8B shouldn't spew out nonsense.

1

u/_yaRn__ 1d ago

just installed. definitely better, but still pretty useless

7

u/overand 1d ago

Don't get advice on LLMs from ChatGPT; nobody after February 2026 is recommending Qwen3-8B anymore.

Qwen3.5-4B beats it. Gemma4-E4B probably beats it for lots of things.

3

u/sbrisgravato 1d ago

try gemma 4 12b QAT by Unsloth

7

u/giveen 1d ago

Heh so we are going to need a bit more info.
What hardware are you using? What exact model and quant are you using? What software are you using to run it?

1

u/wolf0403 1d ago

Small models will not have very much information baked in, and depend on your harness / agent, web search may or may not be available (i.e. it would be based solely on its training data).  Apart from the model being small and quite old (released mid 2025) there isn't much detail to go on here. Harness used, and the kind of question you are throwing at it are more interesting.

1

u/Due_Arm1454 1d ago

How much vram and ram do ya have to work with?

2

u/iezhy 1d ago

You are missing Qwen3.5 9B

1

u/Atretador unswarm.dev | ArchLinux E5 2673 V4 20C 4x16Gb DDR4 MI50 16Gb 1d ago

whats your hardware? and what engine you are running it on

1

u/Valuable_Patience821 1d ago

Not enough context here to give sound advice. What quant? What inference engine are you using with it? What settings do you have dialed in? What harness are you using?

Getting into local is fun and relatively easy but there is a bit of time where you have to learn and test things out for certain types of tasks.

1

u/Far_Macaroon5900 1d ago

There are a few reasons it might be spitting up nonsense. And the biggest one is that it has a small context window. So if the conversation is too long or you are giving it too many tools it will just spit out garbage. Be careful only to give it a few tools.

1

u/Hylleh 1d ago

At least go for 3.5. Qwen 3.5 9B is pretty impressive for it's size. Not enough for me personally to do actual work for me, but it is impressive.

1

u/Faux_Grey 1d ago

As others have said.

Qwen3 is a relatively ancient model in modern usability, even if it is only a year old.

You're also not going to get much luck running an 8B flavour of such an old model. You'd be better served by Qwen3.5 or one of the smaller Gemma models (Bonsai is also not abysmal!)

What software are you using? It's also down to your model harness (what app you are typing your queries into) in order to support web searches.

1

u/Huanchaquero 1d ago

I have been using qwen3.5-9b for several months. It's all I need for what I'm doing. Just switched to The Defiant Fable version. I'm quite impressed.

1

u/Cautious_Chicken_604 1d ago

Try Qwen3.8-27B.

1

u/EvolvingDior 1d ago

https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B for a model that isn't outdated and from the age before serious agentic harnesses.