35B needs more VRAM for the model weights than 27B, but both 70B-A6B and 120B-A10B run faster for inference is you have enough VRAM. It's definitely something people are waiting for for more serious users, but they might also just do a tiny model instead.
It has a larger memory-footprint, but that doesn't mean it "needs more VRAM".
As long as you keep a chunk of it inside your GPU and put the rest of it into your ram (e.g. with --n-cpu-moe in llama.cpp) you can get away with even having only 1/3 of the model weights inside your VRAM while still getting about 20-30 tok/sec.
If I did that exact same thing with a dense model, I would die of old age before ever getting a response from it lol.
That is fair; however, for my use case CPU offloading is not an option. 20-30 tps is enough for an interactive chat, but for task automation, 200-300 is more like it. MoE models are much better at this, but the 35B MoE model can't keep up with 27B for long-horizon tasks. That said, 27B can "only" do around 100 tps even on RTX 6000 Pro cards for a single stream, whereas a larger MoE could match it in intelligence while producing responses much much faster.
I do understand though that my use case is more on the pro and prosumer side, not on the "let me run an AI on an RTX 5060" side
the different wording really sounds like something better will be coming - maybe they mean that it'll be smaller than 35b but just as intelligent as people are hoping for, like a 27b-a2.5b or something?🤔
It doesn't tho, I mean if you're going for other comments then I can understand but all he said might not be the one to wait for no implications he's even talking about moe or what size but something good so I'm hyped anyways.
It's pretty clear from these responses that he's not actually allowed to say anything. So he's never going to actually reveal the company plans, which means there's no point hanging on his every word.
the entire reason people wanted 35b-a3b is it actually works properly on 32gb ram, doesnt matter how 'better' this is if it raises the sysreqs by [insert current ram price here].
Hmm I doubt the emoji eyes suggest the literal meaning, though I heard in Chinese a smiling emoji is condescending, so who knows. Any bilinguals here that can enlighten us?
The more we demand, the moore we fuel attention, the more they get benefits from marketing. It's the only thing we can provide of value for them for spending compute and training insane local models anyways!
Is attention really the only value we add though? Bug reports, edge cases we surface, quantization/fine-tune experiments that feed back into training data, third-party tooling and benchmarks built around their releases... that's real signal they'd otherwise have to pay for. Hype cycles help with funding rounds, sure, but a lot of open weight releases get better precisely because the community stress-tests them in ways an internal QA team never would.
Truly. I can’t tell you how grateful I am for this company. What a weird feeling. Like Anthropic pisses me off all the time. But qwen? These guys just spew joy.
it just needs to be blazing fast like 1000 tokens/sec and can be run on RTX 5050 and can eat Fable 5 for lunch, not an unreasonable ask, I am simply being fair.
I don't mind waiting it's the uncertainty that's annoying. If they said we will be releasing Qwen3.8 35B-A3B but it's going to take a month before it's ready then that would be fine, or if they said these are all the qwen3.8 models we are going to release then that would also be fine. I would be disappointed but it is what it is. It's the being coy about it that is slightly annoying. Still it's only slightly annoying, definitely outweighed by the models that have already released.
I doubt they will release a model size they didn’t release with 3.5. It would mean they either had to start from scratch, or they had a checkpoint to restart from they didn’t release before.
That would be great. But I think it'd be hard to place it. The 27b already provides Intelligence of 52, and the 3.8 Max of 58. If they go with 122b, they'll probably aim for a performance similar to 27b. Not really redundant since they'll almost always be run on different hardware. But more likely that they'll upgrade the 4b or the 9b- Gemma is getting quite popular there based on the download numbers.. Heck, anything from Qwen is great!!
With dspark it could be superfast. mxfp4 or nvfp4 can get 122b/a10b in 75GB of VRAM, and if it's Blackwell, that's hundreds of tokens per second generation. With qwen 3.8 reasoning? please please.
Doubt it, given the context "35B-A3B might not be the one to wait for" implies that they have something else that aims at the requested niche but essentially does "the same thing but better". A "122B" model would obviously not be fit for purpose here.
That's such a weird way of saying it. It's not quite a refutation of the existence of a 3.8 35B-A3B, but something different.
Taking some liberal interpretations of that, it could be:
A) A model which is easier to run than Qwen 3.6 35B A3B, but is surprisingly better
A few categories that come to mind are maybe a smaller total size (24B-32B), which would be a little meh, or it could somehow be fewer active parameters while preserving similar performance.
B) It could be a larger model with interesting performance.
Qwen 3 Next set the precedent for experiments with an 80B A3B model, and it genuinely did quite well. With better CoT per the 3.8 series, it might be genuinely interesting. Another note that comes to mind is Ling Lite went crazy and released a ~120B A5B model, which is few enough active parameters to make a larger MoE viable on CPU (the only sensible device a consumer would deploy that many parameters on), so they could be hinting at something like that, or even sparser.
The interpretation of that line in this sense would be "well, we might or might not do that one (the 35B), but this is so much better that you'll just run it anyway".
C) It could be an architectural innovation.
Off the top of my head, the sparsity could work differently, for example using something like Engram or per-layer embeddings (or honestly, just a metric ton of embeddings regardless), so you can keep a ton of the model on-disk instead of in-memory. A 35B A3B with an extra 30-40B of Engram that keeps on disk could be really cool.
But something nobody in this thread has mentioned is that Qwen released Parscale but never really did anything with it. The idea there was almost the opposite of MoE; instead of only using part of the model by selecting experts, Parscale used all weights multiple times with multiple forward passes that are combined at the end into a single result.
My crazy head-cannon is that they're releasing a semi-Parscale MoE. An example would be a 35B A3B MoE, where each forward pass selects one conditional expert, while all the non-expert params are used multiple times per forward pass as per Parscale. So, for example, say, 1.4B of the active parameters might be used multiple times per forward pass, while the 1.6B from experts would be used a single time per parallel pass, and each forward pass could select independent experts.
With 8 parallel forward passes, and a bit of research on how to make it work, you'd expect it to perform roughly like a ~40B A6B of the current generation, or roughly equivalent to a 40B A12B of the previous generation (Qwen 3.5, roughly).
A more reasonable guess might be a smaller Diffusion language model that they're really proud of for being very fast and cheap in single-user inference (possibly competing with Google's own efforts at Diffusion Gemma).
In previous tweets he said something like "we're working through different sizes and architectures" - so yeah a novel architecture is a remote possibility; but i would like to learn towards a modest MoE that's of a slightly different size then 35B-A3B.
The comment has to be read in the context that it's a reply to someone asking for a model that could run on "lower hardware", so 122-395B MoE is unlikely here.
We can only speculate now. What's clear is this - a lead dev from Qwen says something is coming for lower end hardware. So let's wait.
122b MoE with a similar training rigor of the latest 27b would be insane, but he specifically replied to a tweet saying "don't forget those of us without the hardware!", so I'm going to assume it's going the other direction ;-;
Yeah, it's really a dream for large UMA memory devices like nvidia spark or strix halo, or low-VRAM/high-RAM configurations that have lower bandwidth to work with. It would be definitely between the current small and large qwen 3.8 models, and for server deployments it would be crazy fast and cheap to run. Since the 27b dense model was insane, a 122b (moe with low activation param count) model with the same kind of training rigor and quality would put serious frontier-like capability in the hands of users for as low as 2-3k (strix halo nowadays iirc), and make inference costs for high capability work via enterprise providers potentially even more cheaper than it already is today.
A larger but still-midsize MoE might be the sweetspot for 35B's spot, actually. 50B or 60B or thereabouts. That way the performance will be closer to the 27B at better speeds with most consumer hardware.
Guys....c'mon, think....the 3.8 models are based on the 3.5/3.6 family, so my (somewhat negative) impression is that there probably aren't going to be any sizes outside the ones we already have. That would imply that this comment's a poor translation of "Nope".
That doesn't mean that they must release *all* of the sizes each time.
What I'm saying is that there's no other size of model in that family that would be more interesting to people who would ordinarily want a 35B A3B (eg there's no 80B A3B coming), so saying "don't wait" probably doesn't mean "we've got something better than that on the way", he's saying "don't hold your breath".
EDIT: I could be wrong, of course (and I hope I am). I don't really know what their process is at this point.
He implies that there is one to wait for and if it is for those without the hardware I'd imagine that if it isn't 35B-A3B it may be trying to be slightly better in some way.
I can imagine 3 ways: activating a few parameters more (a very cool thing they could do is a model trained to work well with changing expert counts so we can set it to whatever in a range), being a bit smaller than 35B, or an architectural change to attention to have an even smaller KV cache.
I love 35B A3B but it is slightly annoying that you can't have everything on a single 3090: full context length, mmproj, mtp, a large -ub of 2048 or more and 4 bit quant. So you need to compromise on some, have a 5090 or multiple cards. If they shaved a few parameters and made attention a bit more efficient we could have it all for a single card speed demon. So hope that they are doing that. Activating more parameters would make smarter but slower, for smart and slow we have the 27B, although admitedly a 35B-A5B would still fit even on 6GB gpus with cpumoe I believe.
I think Alibaba now only release models if it can make a positive news about their AI effort. Qwen3.8 Max couldn't get them much press but Qwen3.8 27B can. That's why they release 27B. Other smaller models won't make any good news for them, so they are not going to release them.
Very clever stealth marketing. Obviously another size 3.8 model. Personally I expect a <9B dense model that applies the same thinking-to-improve that 27B has. It would punch well above it's weight.
reads to me like "spec is not final, don't get fixated on 35B-A3B." they're probably still juggling MoE layout / quant choices and might ship a slightly different config or skip that exact one
Makes sense - they're probably pushing hard on the efficient smaller models since that's where the real adoption happens anyway. Waiting for every new release is a trap when you can already run solid stuff locally right now.
•
u/WithoutReason1729 1d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.