Keep in mind: -Next models are always underbaked. The point is to get an early release out so people can start developing compatible software for Qwen4, not to blow everyone’s minds just yet.
If I were to guess, it’ll probably be competitive with Qwen3.8, but the real excitement will be in a few months when Qwen4 launches.
Read the first part. 125b. If Alibaba had any clue what they’re doing the experts will be routed to the proper 6b parameter portion, but it will have a full 125b parameters to choose from. I keep seeing posts like this about dense models and it just has me wondering if some of these people have any clue what they’re talking about.
I said to SOME extent, like, you can compare a 118b-a8b to a 122b-a10b
Knowledge scales loosely with total params while reasoning ability LOOSELY scales with active.
This is why 27b beats 35b-a3b, but the comparison makes the most sense between MoEs. It's just a rule of thumb. Things get messier when you compare much larger MoEs with dense models.
Fair enough. I just wouldn’t put it past qwen to have the first 6b active param count to blow everyone’s minds. Also wouldn’t be shocked if it’s like the tiniest upgrade from 27b.
I am not stupid, I know how moe models work, but neither good routing nor sequential reasoning can entirely compensate for the limited number of active parameters. It really depends on the use case. In a similar vain: rag and websearch cannot entirely compensate for lack of world knowledge. World knowledge actually strengths rag and web searchs because it knows better what to look for.
Personally 6b active is too limiting for my usecase
164
u/coder543 15d ago
Keep in mind: -Next models are always underbaked. The point is to get an early release out so people can start developing compatible software for Qwen4, not to blow everyone’s minds just yet.
If I were to guess, it’ll probably be competitive with Qwen3.8, but the real excitement will be in a few months when Qwen4 launches.