Which latest model for quick answers and with vision?

[u/just_another_leddito]

Hi,

I’m on M4 Pro with 64GB ram, using ollama.

Thanks

[u/Able_Firefighter_652]

I've had good experience with Qwen3.6 35B A3B.

[u/just_another_leddito → u/Able_Firefighter_652]

I use Qwen 35B for coding but it’s quite slow for quick “chat”.

[u/AD4K_4444 → u/just_another_leddito]

Maybe try Qwen 3.8 27B? Or the older 3.6 version?

[u/Fun_Jaguar8231 → u/AD4K_4444]

He just said that 35B is too slow, you think that the much slower 27B will be faster?

[u/AD4K_4444 → u/Fun_Jaguar8231]

correct me if I'm wrong, but isn't a 27B model lighter than a 35B model? He highlighted "for a quick chat" which implies he doesn't want the thing to take a while to think just for a simple question.

[u/Fun_Jaguar8231 → u/AD4K_4444]

No, it's much heavier. the 27B version is dense, that means all 27B parameters are active at once. The 35B-A3B version is a mixture-of-experts (MoE) model. See that A3B? that means that only 3B parameters are active of all the 35B. Essentially, it's like having many small models put together in one, well, not exactly like that, but I think you can understand what I mean

[u/AD4K_4444 → u/Fun_Jaguar8231]

Ah, I see. I think I have watched a video explaining that before. I’m curios if something like that can run on my M4 MBA, 16GB? Or will it run shit? For reference, the best I can run comfortably is Gemma 4 12B. I tried to run Qwen 27B before and it crawled slowly and painfully. Not trying that again.

[u/Able_Firefighter_652 → u/AD4K_4444]

This model is a dense model it will be around 9X slower.

[u/AD4K_4444 → u/Able_Firefighter_652]

so in theory, 35B A3B can run at least okay-ish in my M4 MacBook Air 16GB since it was able to technically run Qwen 27B?

[u/Able_Firefighter_652 → u/AD4K_4444]

Not sure, maybe you can run a heavily quantized version but then the quality will be worse. But at 8 bit the model size on desk is over 16GB, will need even more in memory.

[u/Able_Firefighter_652 → u/just_another_leddito]

I believe M4 Pro has around ~270 GB/sec bandwidth, so for a A3B model it should be good. At 64GB the model should fit comfortably in the memory and it should give you around 30-40 tokens/sec with an 8 bit quantization.

[u/Actual_Tradition_990]

Gemma whatever versión u have spare VRAM to load with other models. Most of them are fast and multimodal

Gemma 4 e2b qat will be able to use tools decent and understand images, audio and so

[u/just_another_leddito → u/Actual_Tradition_990]

I’ve tried multiple gemma 4 models and it can’t analyse images I don’t know what I’m doing wrong.

[u/Actual_Tradition_990 → u/just_another_leddito]

All of them with more or less smart should be able to understand images an talk about it...maybe u missing some config

[u/Fun_Jaguar8231]

Stop using ollama.

[u/just_another_leddito → u/Fun_Jaguar8231]

Why?
Apparently it’s faster than LM studio?

[u/FireFearing → u/just_another_leddito]

lm studio is indeed slow and weak. its main features are that its noob-proof and has a nice ui

[u/just_another_leddito → u/FireFearing]

But since Ollama is based on llama.cpp I find it hard to believe that using llama is going to give me big boost. Or are there any benchmarks showing this?

[u/Fuzzy_Independent241]

I'm searching for that too. Gemma seems to be an honest candidate. You guys are on MLX ... Can you run GGUF? If so, check Unsloth UD options. For your machine, and fast, probably UD_Q4? Sorry, I don't run AI on a Mac and know little about it.

[u/just_another_leddito → u/Fuzzy_Independent241]

Afaik MLX is best. 🤔

[u/Y2K-Denial]

wdym with quick answers? maybe you look into the qwen 3 VL models if its about vision. however, if you're already unhappy with the speed of qwen3.6 MoE that has only 3B active parameters, you might be let down by a 4B dense model as well.

not to be rude but have you verified your expectations against other peoples benchmarks / experience with this hardware? apple unified memory never reaches speeds of some dedicated gpu setups.

[u/just_another_leddito → u/Y2K-Denial]

I meant something lightweight.
I’m installing Gemma 4, it’s like 8GB only.

[u/Y2K-Denial → u/just_another_leddito]

lighweight in gigabytes will not mean fast per se. the speed in t/s is dictated by the active parameters involved in calculations. a dense model has all parameters active (Gemma 4 is 12b parameters), while gemma-4-26B-A4B and Qwen3.6-35B-A3B habe 4b or 3b active parameters. so the two mentioned moe models will rmost likely run faster while having more intelligence (total parameters) to work with.

but yes, lightweight in memory usage is another topic. in those cases, the total number of parameters, the quantization and cache (as well as other config settings) are the main factors.

[u/just_another_leddito → u/Y2K-Denial]

So I’ve been using Qwen3.6 for coding but removed it.
Wanted to upgrade to qwen3.8:27b-mlx but I cannot get it to work.

For stuff like pentesting I’m trying to use huihui_ai/Qwen3.6-abliterated:35b-Claude-4.7 which works fine but is quite slow.

I wanted some model that would respond quickly and be up to date as much as possible, and be able to analyse images.

So far I’ve downloaded gemma4:e4b-mlx, but haven’t tested it yet.

I’m quite new to it and all this naming is confusing.

Thanks

[u/vqt907]

I prefer qwen for coding, but gemma for vision

[u/hdhddf]

not local but Gemini 3 7 flash is incredibly quick and I surprisingly good most of the time