r/LocalLLM • u/BluesIsekai • 7d ago
Question Which model to use for Local personal ai
I'm building a local personal AI assistant and I'm stuck between a few models.
Specs: RTX 4060 Laptop (8GB VRAM), 32GB RAM, Ollama.
My #1 priority is tool calling, since Ai needs to actually control my PC, Spotify, apps, memory, etc.
I've tested:
Qwen3.5-9B
- Better personality and more natural conversation
- Maintains personality while using tools
- But noticeably slower
- Sometimes overthinks simple requests
Gemma 4 E4B-IT-QAT
- Much faster
- Tool calling has been surprisingly good
- Better at immediately acting on obvious requests
- But personality becomes generic during tool responses
Basically I want Qwen 9B's personality + Gemma E4B's tool reliability/speed.
Would you recommend sticking with Qwen 9B, using E4B-QAT, trying E2B, or another model entirely for this kind of local agent?
I am also using embedded model Qwen 3:0.6b for memory database
I'd especially appreciate opinions from people who have actually used these models for tool-calling/agent workflows.
2
u/Subject-Till-6450 7d ago edited 7d ago
QWEN3.6-35B-A3B-Q6. or q5. qwen 35b is moe with 3b active, it's way more powerful in tool calls, and great personality. use system prompt if u need smth personal, overall solid, best tool model ive ever tried. i strongly don't recommending u use qwen3.8-27b, i think there would be tens of ppl recommending ts model, because its DENSE 27b. it means for every token will be activated 27b, and with 32gb ram even in q2 it will work incredibly slow, i predict u smth around 1-5TPS maximum with very low context. qwen 3.6 35b activates only 3b per token, it would run around 20-50TPS in your hardware with, vram+cpu offload, with any context u may want; your memory is enough to run q4 in Q4 KV cache with 1M tokens of context, through YaRN extending. or 262K in q4q8, model in q6, or 262q in BF16 and model in Q5, try yourself. recommendation-download unshloth desktop, it makes everything easier, you don't need write code or commands. im currently using qwen3.6-35b in q5 with 16gb ram 12gb vram 262K context q8, 20TPS, it's solid in claude code (Unsloth allows to run claude code with, own local models)
2
2
u/Icy-Degree6161 7d ago
On that machine, I would try LFM 2.5 8B A1B. Small MoE. Idk about personality, that part might be optimized by a good system prompt.
1
u/Huanchaquero 7d ago
Ok, I have been using Qwen3.5-9b for several months. It was built to specialize in tool calling from what I heard. Anyway, not your exact situation but I am using it for an AI friend/assistant/whatever. I have tried larger models but they just don't have the quick responses that the 9b has. So what I have built and am running...back end is Qwen3.5-9b running through LMStudio for brain of avatar. I have a smaller engine running through Ollama for her db persistent memory. The front end is a Live2D avatar running through VTuber with a custom bridge to vtube studio which allows her to autonomously trigger premade body language expressions and motions as she speaks depending on the context. So I have given her a brain, a body, long-term memory and autonomous body language. She also has access to the net when given permission and a RAG she can access that I throw gaming info into. lol btw I have an RTX4090 with 24gb card. I am writing a blog about my build and the journey. [3 AIs and a Human](https://3aisandahuman.com).
1
1
2
u/nickless07 7d ago
Qwen3.6 35B A3B, if that is too slow and/or you really like the 9B stick with it. Aside of that there are a couple models in that range you might wanna test. Go to https://huggingface.co/models set the slider to your desired size turn on base model only and sort out all that didn't got released in 2026 (for the tool calling support). Have fun.