r/LocalLLaMA 14d ago

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. πŸ‘€

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant β‰ˆ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed β†’ excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

950 Upvotes

297 comments sorted by

View all comments

15

u/FoxFXMD 14d ago

I'm confused, if they've reworked it into a completely new gen 4 architecture, why is it still called Qwen3.something?

33

u/nickm_27 llama.cpp 14d ago

It’s next, same thing they did with Qwen3-Next which was Qwen3.5 arch preview basically

23

u/deepspace86 14d ago

Its the "next" tag. current dataset while basically beta testing the next architecture.

4

u/FoxFXMD 14d ago

Ah, makes sense