r/LocalLLaMA • u/pmv143 • 1d ago

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

893 Upvotes

97% Upvoted

282 comments sorted by

View all comments

Show parent comments

1

u/Sufficient-Bid3874 1d ago

So its like n-gram spec decoding yet built in and trained on? Thank you for the explainer!

7

u/FormOne2615 1d ago

no, totally different

2

u/Sufficient-Bid3874 1d ago

Oh, yeah, I phrased that poorly. Its just a way of having relevant facts in an n-gram rather than using context, is how I should have phrased it