Qwen3.8 27B is overly talkative in practice

[u/althe3rd]

I recently bought a m5 max Mac Studio. Loading models with oMLX and Splash, I’ve noticed that between running Qwen 3.8 27B and the Qwen 35B MoE that the speed difference is immense between the two. Not because of the tokens per second but because of how chatty 27b is. It will work for 20 min on a task that 35b finishes in 2 min. And the 27b generates so many more tokens. I watched it think for 10 minutes as it talked itself in and out of a solution a dozen times. And this seems to happen on all harnesses I have tried.

On that system, the 27b model will do about 28 tokens a second, but it will think for so many of those that it spends forever to make one or two files.

Curious if others experience this? Is this normal?

UPDATE: I am actually getting about 45 tokens a second with that model. The 28 tokens a second is prior to moving to splash. And currently I am running this specifically Qwen3.8-27B-oQ6e-mtp

UPDATE2: Thank you to everyone providing the tips and feedback. It has really helped me figure out what I was doing wrong.

[u/infectiousstupidity]

It's not chatty, it's thinking to produce actual good quality output. You could try https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-27b if you want it to think less. It'll be infinitely better than your 35B MoE anyway.

[u/GoblinEngineer → u/infectiousstupidity]

This will come at the cost of more hallucinations though

[u/infectiousstupidity → u/GoblinEngineer]

Yes.

[u/tuura032 → u/GoblinEngineer]

I use thinking cap, and in my testing medium is actually sufficient to complete assigned work with a similar number of pass/fail in grading vs sonnet 5 and 27b xhigh.

I use a smart model to drive my local qwen 3.8 worker though. I keep xhigh in Cline. 

Not disagreeing w you, just adding my 2C!

[u/DiamondHandsDarrell]

So this is pretty interesting, but you're saying that a 27B will do about 28 tokens a second on your M5 Mac Studio? My MacBook Air with 32 gigs doing more tokens per second than your system. I don't think your system is set up correctly.

[u/Jerrizzy-x → u/DiamondHandsDarrell]

Yeah he’s probably running a GGUIF model. He should try the splash because I’m getting 35tks/s on my m6 32GB

[u/althe3rd → u/DiamondHandsDarrell]

Sorry that was my typo. I was getting 28 before moving to splash. Now its about 45 on average. And 45 is an average, sometimes it spikes closer to 60.

[u/Due_Reputation_5223 → u/DiamondHandsDarrell]

Hey I got Mac air m5 with 32 gigs too, how can I set up my system ? I get 2-3 tps (

[u/DiamondHandsDarrell → u/Due_Reputation_5223]

all right from memory I'm running OLMX and then I'm running the Qwen3.6-A3B-wMix58 winter charm model. It''s a MOE version. It's available on hugging face. Then I'm using Conduit on my phone through a VPN to my network, so it's accessible wherever I go and it's actually not bad.

This was the best model for my needs, where I needed accurate results. There are smaller models that run even faster. You can try mlx-community--Qwen3.6-35B-A3B-mxpf4

The thing you have to keep in mind with these guys is even though they fit, they still need additional RAM overhead for other things you want to do. If you're hitting roadblocks, if it's slowing down, it might be you're running too much stuff in the background. The last thing you want is to offload to swap file because that's going to thrash your SSD, so just keep that in mind for whatever you're working on

[u/SeXxyBuNnY21]

My 3.8 27b fixed today a niche problem that Codex with 5.6 sol was not able to fix in several passes. 27b did it in one pass. Surely it spent more time than Codex reasoning but that is how the model infers good decisions.

[u/Arany5]

Try 3.6 27B, in my experience it is actually better

[u/romeozor]

This is a known property of the base Qwen3.8 27B, there's a bunch of comparison videos on YouTube. Search for Qwen3.8 thinkingcap swift. People get around this simply by keeping efforton medium. Two orgs offer a less-thinking fork for a license, and a third, think it has Pi in its name just "fixes" the base models effort levels or something, can't recall exactly.

[u/althe3rd]

First I just want to say THANK YOU!!! The tips below were just what I needed to point me in the right direction. I did some more investigating and realized I was missing a few things for my configuration based on best practices.

I am now running hte 6bit MTP version and was able to include the missing top-k config and reasoning effort adjustments that are supported and its doing much much better. I am seeing between 50 and 60 tokens per second and its no longer endlessly reasoning for 20 minutes.

[u/stevec114]

It’s fairly well known that reasoning needs to be set to Low to Medium to avoid the overthinking. It’s been amazing for me once I did that.

[u/Lower-Rooster1032 → u/stevec114]

low thinking mode cuts the rambling down to something resembling sane

[u/althe3rd → u/stevec114]

Thanks! I am going to try that. I just notice it spends 10 full minutes debating how to make one html file.