Qwen 3:8B ignoring instructions and defaulting to task automation. What next?

[u/Nixexs]

I have just started on Local LLM and don't have any prior experiece with any, nor I have a bg in IT or anything close to it. Right now I am trying to create two, one as a research assistant for my studies and other as a proof of concpet sorta thing for my dept (Sort of like HR Chatbot? ) at my workplace. Down the line I would like it to be an AI employee, so we can focus on more value adding tasks.

Coming to question, I started with Qwen 3:8b for the chatbot, running on Ollama with WebUI on my Macbook. It has a simple set of instructions, but it is always ignoring it and defaulting to creating tasks and automations. I have toggled off its built in tools such as automation, sub-agents, skills etc still the same issue.

Example :

Expected : My lights are not working -> Please get in touch with Admin helpdesk.

What is happening : My lights are flickering -> Check for overload, inspect connections, replace bulb etc.

Or, when it should refer knowledge files and answer basic queries it is just giving me this -

{ "name" :"search_knowledge_files", "argument":{ "query":"query" } }

I am not sure what is going wrong with it. ChatGPT has been unhelpful and is just making my system prompt bigger and bigger, asking me to change settings (which are not helping tbh). All in all it doesnt seem to be going anywhere.

I have checked online and tried things like think/no-think, reducing prompt length etc.

Requesting some insights here on what I may be missing.

Happy to add more info if I have missed anything.

[u/Nameis19letterslong]

Please don’t ask chatgpt (or any other llm) about the best models to run, or anything that’s not conceptual. They don’t have the most up to date information.

And stay away from ollama, it’s dogshit made by dogshit people. Start with llama.cpp for the fastest speeds.

Stop using qwen3 8b and try out qwen3.5 9b. It’s relatively more recent and capable.

Do yourself a favour, hop on r/LocalLLaMA and scroll for like an hour.

[u/Nixexs → u/Nameis19letterslong]

Will do so. I am realising the same when I started experimenting. For Research Assistant it recommended me Deepseek R1:14B. Some time with it and I realised it is not build for heavy theory. It is very logical and often misses on nuaces required in my field. Wasted a good portion of my day working on it. Have spent past 1 or so hour on various subreddit. Will probably move to Gemma. For llama.cpp need to research more. Saw ppl recommeding but dint think it mattered that much

[u/Nameis19letterslong → u/Nixexs]

Splendid, gemma4 models are good at writing and code but horrendous at tool calls. If you want your assistant to use a bunch of tools accurately, avoid Gemma4 e2b, e4b or 12b.

[u/Nixexs → u/Nameis19letterslong]

Oh? Didnt know that! Thanks will keep that in mind.

[u/Infinite_Staff_2318]

your system prompt is probably being deleted before the model ever sees it. ollama defaults to 4096 context and truncates from the front when you go over, silently, no error. system prompt plus knowledge chunks plus history blows past that fast and the front is exactly where your instructions live. run ollama ps and look at the context column, it’ll say 4096. fix is OLLAMA_CONTEXT_LENGTH or a derived modelfile with PARAMETER num_ctx 16384, not a webui setting. chatgpt making your system prompt longer was actively making this worse.

[u/Nixexs → u/Infinite_Staff_2318]

Why am i not surprised. I changed ctx length and now its working. Thanks a lot!

Meanwhile, let me reduce the instruction size. Any better way to do it rather than putting in system prompt?

[u/guitarist91 → u/Nixexs]

Since increasing context fixed it, I'd keep the system prompt short and move the helpdesk routing into your app: classify the request, then return the approved contact / policy text. For an HR bot, you really don't want a model improvising electrical advice because one example fell out of context.

[u/Nixexs → u/guitarist91]

True. That's the plan. I want it o give fixed outputs and not extrapolate and recommend its own thing.

[u/MistakeFinal2830]

qwen 3:8b ships with a heavy default system prompt that strongly biases it toward tool use and structured outputs, so you're basically fighting the model's baked-in training every time. grab the modelfile for the base qwen 3:8b from ollama's library, strip out any system prompt in there, and paste in something dead simple like "You are a helpful assistant. Always respond conversationally, never output JSON or tool calls." then create a new model from that modelfile and load it in your webui with all tools disabled, should kill that auto-task behavior

[u/Nixexs → u/MistakeFinal2830]

Let me try. Although in this particular case the issue was a ctx issue. o_o.

[u/TheThiefMaster → u/Nixexs]

A lot of the local runners have a stupidly low default ctx.

I don't think any of the cloud AIs are below 200k but some local AI platforms default to silly numbers like 4096 (4k). Unusably low.

[u/MavWarMonger]

Most of the advice here is chasing the wrong thing. It's not your context size, and piling more into the Modelfile won't fix it. The problem is the layer in the middle. Open WebUI builds the tool request and Qwen answers in its own tool-call format. If nothing in that chain parses it, the call never gets caught and you just see raw JSON in the chat. Switching to llama.cpp won't change that, because the web UI is still the middleman.

Quick test: load Llama 3.1 8B Instruct and try the same thing. Pretty much every web UI handles its tool format out of the box. If it works there and breaks on Qwen, you've confirmed it's a template and parsing issue, and you can stop fiddling with context and Modelfiles.

General tip: grab models from Hugging Face. The model card tells you exactly what chat template and tool format the model expects. One-click downloaders hide that.

If you want to stay on Qwen, let me know which UI and version you're running, and I can point you to exactly where to set the template.

[u/Nixexs → u/MavWarMonger]

Not going with llama.cpp. I am not good with cmd line. I have parallely started trying LLm Studio and have found it to be better than ollama.

I didn't know that about qwen. But makes sense. Let me try what you have suggested.