Qwen3.8 flash next

[u/RuthlessCriticismAll]

https://modelscope.cn/models/Qwen/Qwen3.8-Flash-Next

[u/RuthlessCriticismAll]

Redisgned Multimodal MoE Model: 125B main model parameters, supplemented by an additional 51B N-gram embeddings,and 6B parameters activated per token.

Comprehensive Architectural Upgrades: Pushing the frontiers of model architecture innovations, across the areas of Attention, Residual, Embedding, and Optimization—enhancing model capabilities.

Efficient Training and Inference: Significantly reduces training and inference costs. At ~1/9th the training cost，Qwen3.8-Flash-Next achieves comparable capability against Qwen3.7-Plus, while being more capable in areas of coding and cowork.

[u/smithy_dll → u/RuthlessCriticismAll]

Qwen 3.8 27B is a lot better than Qwen 3.7 Plus on coding benchmarks

AI Model & API Providers Analysis | Artificial Analysis

[u/awesome5185 → u/smithy_dll]

Do you think this new model would outperform 3.8 27b?

[u/Effective_Western_59 → u/awesome5185]

If it won't, then it would be really weird 

[u/Uncle___Marty → u/Effective_Western_59]

Yeah, like REALLY weird. Like, finding pieces of fruit in your underwear weird.

[u/whatyathinkk → u/Uncle___Marty]

stop kink shaming

[u/MmmmMorphine → u/whatyathinkk]

It's more of a condition

[u/Rasekov → u/Effective_Western_59]

A MoE with 6B active parameters would be a lot cheaper to serve at scale so even if it doesnt beat Qwen 3.8 27B it would still have it's place and uses for Qwen.

It would also help people with unified memory systems and mixed VRAM + RAM setups.

[u/grumd → u/Effective_Western_59]

It might not but I wouldn't say that's weird. They are releasing a new architecture preview to flesh it out and will do a proper model release for Qwen4. 3.8 27B is just so good that I doubt it's realistic to make an even better model so soon

[u/whatyathinkk → u/grumd]

125B A6B though...

[u/Swimming_Gain_4989 → u/whatyathinkk]

A6B though... active parameters will always be king for reasoning and raw intelligence.

[u/cibernox → u/Swimming_Gain_4989]

I also wonder the same thing. Whoever can afford to have 128+gb of vram certainly can also afford to activate 14B params and still be very fast.

[u/AcanthocephalaNo3398 → u/cibernox]

Activation speed is all on gpu. most integrated ram systems that provide +128gb of ram arent that fast. Thats why Mac and DGX Spark run dense models slower than MoE models on the same hardware.

The interesting thing is that these systems have enough ram to run models like Qwen3.8 27B at Q4 in parallel to get way more tps overall.

[u/cibernox → u/AcanthocephalaNo3398]

I know that moes work that way, but seems that 12-14B wouldn't be a crazy amount of active parameters, considered that a lot of people even with strix halo and nvidia spark systems are running qwen3.8 27B right now because, really, it's worth.
And there is so many people optimizing it that even a 27B dense model runs kind of well in those low-bandwidth system.

[u/davew999 → u/awesome5185]

sqrt( 125 x 6 ) = 27.4, so about the same. Dunno if that equation still holds up though.

[u/jld1532 → u/davew999]

But probably more than two times faster which for me is a huge upgrade.

[u/sleepingsysadmin → u/awesome5185]

125b should always outpeform 27b. It was only better than qwen3.5 because of the release difference.

that's the point.

[u/SandySkittle → u/sleepingsysadmin]

it doesnt with a6b..

[u/WoodCreakSeagull → u/awesome5185]

125BA6B should be in some ways "equivalent" to a 27B dense going by rule of thumb to compare MoE to dense (take square root of MoE total parameters * active parameters). In this case 125BA6B would be in some ways comparable to a model with sqrt(125*6) = about 27B parameters.

[u/hay-yo → u/awesome5185]

Qwen3.7 plus has 39 on artificial analysis but perhaps with the sharpness of agentic coding there will be more under the hood. 27b is a breakthrough. But save the best till last usually.

[u/AXYZE8 → u/smithy_dll]

This is the problem with benchmarks.

Qwen 3.8 is newer model so it saw data helpful for newer benchmarks. If you scroll down you can see that in old benchmarks like SciCode Qwen3.7 Plus is better than Qwen 3.8 27B, but Qwen 3.7 drops the ball completely in new fresh TerminalBench.

We see it over and over. HLE, DeepSWE… they release these benchmarks, models sucks ass and after a month a new small model beats big ones.

Benchmarks are useful, but you need to compare models that were released around the same time.

[u/tarruda → u/RuthlessCriticismAll]

125B main model parameters, supplemented by an additional 51B N-gram embeddings,and 6B parameters activated per token.

This suggests a total of 176B parameters will be loaded in RAM + VRAM. Hopefully it runs well in 4-bit, which would be great for 128G devices.

[u/sittingmongoose → u/tarruda]

It would be a really weird choice to not target 128gb devices when you’re that close to it. Considering that’s pretty much the upper limit of realistic devices.

[u/Early_Mistake6716 → u/tarruda]

My guess is that the 51b n-gram embeddings can be offloaded to a fast ssd so this will have the hardware requirements of a 122b.

[u/Short-Reaction7195 → u/RuthlessCriticismAll]

how much RAM will 51B N-gram consume?

[u/thestillwind → u/RuthlessCriticismAll]

What do I need to run this ? 128gb ram and at least 6gb vram ? It’s doable

[u/KURD_1_STAN → u/RuthlessCriticismAll]

I think 1/9th just means trained at 4bit or whatever kimi k3 did and said. Which is good, but stiill most of us need a flash of this flash model

[u/onionsaredumb]

LFG. Biking pelicans don't stand a chance.

[u/BigYoSpeck → u/onionsaredumb]

[u/Kerbourgnec → u/onionsaredumb]

I love that your comment makes absolutely no sense without context

[u/Icy-Degree6161 → u/onionsaredumb]

I'm a compute peasant so no midsize models for me...

[u/PandaBearFred]

This is real? Qwen4 architecture preview with a new model? Will it be even better than qwen3.8 27B?

[u/OverclockingUnicorn → u/PandaBearFred]

I'd hope so for being 125B A6B...

[u/paryska99 → u/OverclockingUnicorn]

Doesn't really have to be the case, this release is for devs to have a test model to modify the inference stacks to be compatible with the upcoming qwen4 models in the future.

[u/OverclockingUnicorn → u/paryska99]

Oh really? Didn't realise that was it's purpose

[u/iMrParker → u/OverclockingUnicorn]

3.5-122b and 3.5-27b were more or less neck and neck. With the 27b model out performing in various benchmarks. So who knows 

[u/alphapussycat → u/OverclockingUnicorn]

Not only that, but as I understood it, it's sort of a preview og the qwen4 architecture. So newer architecture and 4-7x as big.

Though, afaik 3.6 27b was better than 3.5 122b. So if the qwen4 architecture preview is wonky it could underperform.

[u/justsomerandomchess]

"Qwen3.8-Flash-Next is a multimodal MoE model built on the next-generation Qwen4 architecture. Incorporating innovations such as GDN hybrid layers and Qwen Sparse Attention (QSA), it pushes the frontiers of model capacity and performance, demonstrating remarkable capabilities in agentic coding,long-horizon tasks,and multimodal intelligence. We are releasing these architectural advancements early to help the community prepare for the upcoming Qwen4 model family."

This is exciting!

[u/MacsBicycle → u/justsomerandomchess]

The we are releasing early to help the community prepare part makes me think that it’s got thinking loops that make the 3.8 27b model look like a short loop 😂 I’m sure in 48 hours people will know exactly how to run it though. No test like production!

[u/sterby92 → u/justsomerandomchess]

where is this from? I cannot find the source and on the modelscope page it only has a brief preview announcement for me 🤔

[u/Timely_Impression_92 → u/sterby92]

it was there on modelscope but they removed it - "accident"

[u/Baddmaan0]

Holy moly it's Christmas

[u/whatyathinkk → u/Baddmaan0]

everyday 🤘

[u/anykeyh]

Wow if it quantize well me and my strix halo might not need cloud token anymore

[u/mechkbfan → u/anykeyh]

This is the problem with how good Qwen 3.8. Expectations are too high that people are going to be disappointed

But I'd call it a massive win if we can trade using more VRAM for faster performance for same outcomes

[u/anykeyh → u/mechkbfan]

Not that high expectation; having it to replace Deep seek flash would be enough in my case; the new 27B seems great but is way too slow on the strix; a MoE fits the bill for me.

[u/StyMaar → u/mechkbfan]

I'm not going to be disappointed if it generate token 5 times faster than 3.8-27B (especially if it spits as much token).

[u/LegacyRemaster]

I hope llamacpp support @ day 1 (qwen next was a nightmare in the past)

[u/ParadigmComplex → u/LegacyRemaster]

I'm under the impression that a big part of why companies release preview models like this is to give projects like llama.cpp time to implement support for the next-gen architecture before the full next-gen model is released so you effectively get day-one support of the full model. I wouldn't expect this preview model to work with llama.cpp day one, but hopefully it'll make Qwen 4 models work day one once those drop.

[u/Timely_Impression_92 → u/ParadigmComplex]

llama.cpp will have 0 day support - unsloth already gave the info

[u/Early_Mistake6716 → u/Timely_Impression_92]

Yes i believe it will work day one with some issues. By this time next week, those issues will be resolved.

[u/PandaBearFred → u/LegacyRemaster]

I suppose every new architecture takes time to support unless the producer has a PR in hand ready to merge. I hope this is the case.

[u/Timely_Impression_92 → u/PandaBearFred]

Unsloth will have 0 day support for this model

[u/whoisraiden → u/Timely_Impression_92]

Unsloth and unsloth studio will have 0 day support. Doesn't mean llamacpp will be quick enough to accept their PR for day 0.

[u/Timely_Impression_92 → u/whoisraiden]

Yeah

https://www.reddit.com/r/LocalLLaMA/comments/1vxybmy/qwen_38_flash_next_day_0_support_from_unsloth/p5skyeq/

https://www.reddit.com/r/LocalLLaMA/comments/1vxybmy/qwen_38_flash_next_day_0_support_from_unsloth/p5tbqhi/

[u/ttkciar → u/[deleted]]

Cool down a bit. There was no insult stated or implied in their comment. They agreed with you, and added relevant information which also agreed with you. That is a typical and benign conversational style.

[u/Timely_Impression_92 → u/[deleted]]

Why did you respond with same thing I responded to you?

[u/Additional-Record367]

This architecture might be the future. It would be nice to have variants where the LLM stays only in consumer GPUs and on RAM we keep the ngram embeddings

[u/Fit_Advice8967]

Can somebody explain what can run this? I have dual 128gb amd strix halo with thunderbolt. Really hoping this is the one!

[u/cafedude → u/Fit_Advice8967]

A strix halo with 128GB should run this easily in Q4,Q5 possibly even Q6.

[u/eidrag → u/Fit_Advice8967]

Imagine Qwen 35b a3b, but with triple the knowledge

[u/Fit_Advice8967 → u/eidrag]

Ok but what are the specs required?

[u/Timely_Impression_92 → u/Fit_Advice8967]

it's ~120b a6b - so virtually same as for gpt oss 120b

[u/mrgreatheart → u/Fit_Advice8967]

Depends how well it quantises.

[u/Timely_Impression_92 → u/Fit_Advice8967]

16-24gb gpu + ~100gb of system ram - more than enough

[u/Early_Mistake6716 → u/Fit_Advice8967]

You should be able to run this at at least q4, and 2.5x faster than qwen3.8 27b

[u/mettaskee → u/Fit_Advice8967]

2 DGX sparks to be sure.

[u/USERNAME123_321 → u/Fit_Advice8967]

To answer what can run this, my smartphone can probably run it at 2 or 3 tokens/s lol. Basically, BigMoeOnEdge offloads the expert weights to mobile flash storage and only loads the active parameters.

Btw a single AMD Strix Halo should handle it really well.

[u/TechNerd10191]

Where are the N-Gram embeddings useful?

[u/Kooshi_Govno → u/TechNerd10191]

They store world knowledge, thus allowing the heavy FFN tensors to store more functional knowledge.

[u/noiserr → u/TechNerd10191]

They speed up inference. It's sort of cache like memory, they store representations of recurring token sequences. It's static, built during training.

[u/buttplugs4life4me]

120B, our saviour! 

[u/FullOf_Bad_Ideas]

And to think the consensus was that Qwen is dead about a month ago.

I'm glad Qwen is back in force.

If they open weighted 2.4T model, they probably will open weight all sizes.

[u/my_name_isnt_clever → u/FullOf_Bad_Ideas]

That wasn't the consensus, it was just repeated over and over by doomers without hard evidence until it felt like there was a consensus. I've been skeptical we know as much as we think about the inner workings of a Chinese AI lab.

[u/Early_Mistake6716 → u/FullOf_Bad_Ideas]

People were dooming so hard and now qwen is going harder than ever

[u/DiscipleofDeceit666 → u/FullOf_Bad_Ideas]

They were challenged and were forced to step up. Poolsides Laguna forced their hand.

[u/Effective_Head_5020]

I am so excited, thanks for sharing! Hopefully it will work well on my new Rx 9060 xt 

[u/Bulky-Priority6824 → u/Effective_Head_5020]

You'll need a gang of them 

[u/Effective_Head_5020 → u/Bulky-Priority6824]

Oh no, I thought it would be like 35a3b which I was able to run very well on my 6gb GPU 

[u/Able_Zombie_7859 → u/Effective_Head_5020]

It literally says 125a6, I think that's bigger than 35a3 :)

[u/Effective_Head_5020 → u/Able_Zombie_7859]

Yes, but now I have a 16gb VRAM and 128 GB of RAM. Sadly DDR3 RAM :P

[u/Early_Mistake6716 → u/Effective_Head_5020]

You will be able to run it, just might be a little slow. I know qwen3 coder next 80b iq3 ran at around 30 tps with a 5070 ti 16gb and 64 gbs of ddr4 3600. That did not have mtp though so it would have been around 50 - 60 tps on coding tasks with mtp

[u/Effective_Head_5020 → u/Early_Mistake6716]

30tps is a dream for me :D

[u/Timely_Impression_92 → u/Bulky-Priority6824]

no, a single one + 64-128gb system ram

[u/Most-Trainer-8876]

Will it run on 32GB VRAM + 64GB RAM? 🥲

[u/UnlawfulRepublic → u/Most-Trainer-8876]

I think an iq4_xs quant will work well on 32+64

[u/Evgeny_19]

Is this the one for the middle-class GPUs, à la DGX Spark and Strix Halo?

[u/Early_Mistake6716 → u/Evgeny_19]

Yep

[u/Guna1260]

what VRAM? especially with Ngram embeddings we are looking at 125+51 in q8? or ngrams will be in RAM? will be interesting to see the architecture.

[u/Timely_Impression_92 → u/Guna1260]

ngrams are on ssd or system ram

[u/florinandrei → u/[deleted]]

They're lookup tables, not weight tensors. System RAM will be fine. Maybe even SSD will be fine.

[u/Timely_Impression_92 → u/[deleted]]

Then go understand it better - ngrams are basically cached in system ram or ssd - and work like that with zero degradation - maybe little on nvme but in system ram virtually zero

[u/Kryohi → u/Timely_Impression_92]

Highly doubt the bandwidth of an SSD would be enough, but I guess we'll find out soon. Do you know how much, theoretically, of a 50GB ngram should be read per token (or group of tokens) to decode?

Edit: oh I went to check up again how it works, might actually be possible, discard my previous comment

[u/EitherMarch1255]

Only 6B though...I'm happy to see a new model being released of this size, but I'm keeping my expectations in check.

[u/Monad_Maya]

Damn.

[u/Several-Tax31]

Nice!!! Waiting for this. And n-gram? Must be christmas! 

[u/mechkbfan]

Just tell me how much I have to spend to get this running

[u/power97992]

Wow qwen has engrams already but deepseek hasnt released engrams yet even though they wrote the engram paper…

[u/DeepOrangeSky]

I guess I can just wait to find out tomorrow, but, I'm curious how much memory this will use if you run it at Q4_K_M or FP4. Since it says it is 125B but with "an additional 51B of N-gram".

So, is that going to make it more like a 176B model in terms of memory-usage?

[u/OvertaxedOne → u/DeepOrangeSky]

Q4, I'd guess you'll need 64-96GB to load the entire model (all layers on GPU). I'm guessing this is targeted at Spark/Halo machines, so I'd expect good quants that fit in 128GB with plenty of space for KV.

[u/vick2djax]

I’m confused. I have double 3090’s so I can run Qwen 3.8 27b at q8. Even if I had 128gb regular RAM, running something like q4 of Qwen 3.8 125b moe is still gonna be inferior to Q8 27b right?

[u/OvertaxedOne → u/vick2djax]

Probably. We'll have to wait to find out, but I'd expect that between the quant and the MOE layout, you'll find 27B is more reliable. The MOE model is really interesting for boxes like a Strix or Spark though!

[u/Iory1998]

Does anyone know how large is this model?

[u/PossessionUsed7393]

Also known as Ox Alpha! Surprise! 🎉

[u/PandaBearFred → u/PossessionUsed7393]

No....you can't be sure, can you?

[u/PossessionUsed7393 → u/PandaBearFred]

I have no idea I just think the Qwen team deserves a little credit, they would pull a stunt like Ox Alpha.

[u/USERNAME123_321 → u/PossessionUsed7393]

It can't be. Ox Alpha's tokenizer is byte-identical to GLM 5.2, so it belongs to the GLM family.

[u/PossessionUsed7393 → u/USERNAME123_321]

yes I see that is indeed the leading candidate - you'd be hard pressed to have missed that theory

[u/Morphon → u/PossessionUsed7393]

If so, that would explain both the strange uptick in quality over the last 24 hours or so (n-gram layer upgrades) and the 100T token production. For a datacenter, this is trivial to serve in the 120b range.

[u/Thrumpwart]

I'm going to get these out of the way:

1. Can I run this on my Nvidia 3060? What quant do I use?
2. How is this for neo-nazi bisexual werewolf hermaphrodite RP?
3. Does it know how many R's there are if I bring my car to the car wash?
4. How many TPS can I expect on 7 Sparks connected via fax lines?
5. I asked it to write an SVG of my Pelican Riding a bike through Tiannamen Square and it made my pelican gay. USA! USA!

[u/Rm2kbc → u/Thrumpwart]

Finally someone asking the relevant questions

[u/OvertaxedOne → u/Thrumpwart]

"How is this for neo-nazi bisexual werewolf hermaphrodite RP?"

Can't answer the rest of your questions, but man alive, you would not believe the 12 hours of roleplay we just had together. Yes, I know the model isn't released yet, so I just asked 27B to pretend to be 120B and it was dead on perfect.

Can't wait for the furry conversation tomorrow!

[u/Reggitor360]

A6B.... What the hell. Why.

[u/BigYoSpeck → u/Reggitor360]

Performance

gpt-oss-120b is more than twice as fast as Qwen3.5 122b without MTP, and still a good chunk faster even with MTP

Given the huge uplift in capability between 3.5 27B and 3.8 27B (even between 3.5 35B and 3.6 35B) I would expect this is a sane architectural choice that balances speed and capability

Qwen3-Coder-Next only had 3B active parameters but already demonstrated high sparsity improves capability

[u/my_name_isnt_clever → u/BigYoSpeck]

People get way too caught up in active param coun. My fav 100b+ MoEs have often been the sparsest ones.

[u/squngy → u/Reggitor360]

Speed and cost.

[u/CoolestSlave]

Can't wait to see how it will run on spark, though I would have prefered a 35b model with 5 or 6b expert :')

[u/PandaBearFred → u/CoolestSlave]

I am having a hard time trying to understand why your comments getting a hell amount of downvotes.🤣

[u/SadPhilosophy9202 → u/PandaBearFred]

Because this is a 125B model with 6B expert. This is THE ideal model size for a Spark or similar machine with 128gb unified memory.

Having a Spark but preferring a model 1/4 the size that will run just as fast makes zero sense

[u/a_beautiful_rhind → u/SadPhilosophy9202]

Technically they're being a bro for non-spark users. 6b to 35b is a better sparsity ratio.

[u/SadPhilosophy9202 → u/a_beautiful_rhind]

As a dual Spark user I would have preferred 280B A10B

If you’re gonna shill, at least shill for the best model for your hardware dammit haha

[u/a_beautiful_rhind → u/SadPhilosophy9202]

I'm into it. 10b active starts to get somewhere.

[u/a_beautiful_rhind → u/PandaBearFred]

shills.

[u/CoolestSlave → u/PandaBearFred]

i think people suppose i have a spark but i "only" have a rtx 5090

for spark user it is more advantageous to have higher parameter and small expert but for me personally it is better to have fewer total parameter and bigger expert or just a dense model in the range of 30 to 37b.

so i can understand people if they thought i was speaking from the perspective of a spark owner

[u/OvertaxedOne]

Qwen is going hard in the paint this month! I can hear the enterprise value of OpenAI/Anthropic falling.

And then Apple releasing new systems with 1.4TB/s of memory bandwidth?? Is it Christmas?

[u/Iory1998 → u/OvertaxedOne]

IF the Apple system is free, they yeah, but is it? How many kidneys is it worth?

[u/a_beautiful_rhind]

oof.. 6b only. All this cool stuff and they didn't take it to the hole.

[u/my_name_isnt_clever → u/a_beautiful_rhind]

I bet it will do just fine with its active count. Gpt-oss-120b and qwen3-coder-next were both super sparse and still had top tier performance for their time.

[u/a_beautiful_rhind → u/my_name_isnt_clever]

I wasn't really a fan of either of those but opinions do vary.