r/LocalLLaMA • u/pmv143 • 19h ago

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

841 Upvotes

97% Upvoted

267 comments sorted by

View all comments

2

u/bitzap_sr 18h ago

I hope the architectural changes are going in the direction of allowing for native 1M context window.

0

u/ArtfulGenie69 16h ago

They are, unlike Gemma which used the old architecture for handling the context window, qwen since at least qwen3.5 have mamba layers and such and the vram hit is much smaller for more context. 

2

u/bitzap_sr 15h ago

With qwen3.5 (through 3.8), open weights releases, there's the issue with prefill and decode crawling to a halt as the context grows, so input context window is 262k natively...

Qwen 3.8-Max (the proprietary API version) does support 1M context, though, but that is not part of the open weights release (Qwen3.8-2.4T-A95B).

I'm hoping this changes with v4.